Running the Deep Memory Training

Now we can run a Deep Memory training, which runs asynchronously and executes on our managed service.job_id = vectorstore.deep_memory.train( queries = claims, relevance = relevances, embedding_function = embedding_function,)All of the Deep Memory training jobs for this Vector Store can be listed using the command below. The PROGRESS tells us the state of the training job, as well as the recall improvement on the data.recall@k corresponds to the percentage of rows for which the correct (most relevant) answer was returned in the top k vector search resultsvectorstore.deep_memory.list_jobs()This dataset can be visualized in Jupyter Notebook by ds.visualize() or at https://app.activeloop.ai/activeloop-test/test-deepmemory-ivoID STATUS RESULTS PROGRESS 6525a94bbfacbf7e75a08c76 completed recall@10: 0.00% (+0.00%) eta: 45.5 seconds recall@10: 0.00% (+0.00%)6538186bc1d2ffd8e8cd3b49 completed recall@10: 85.81% (+21.78%) eta: 1.9 seconds recall@10: 85.81% (+21.78%)


Evaluating Deep Memory's Performance

Let's evaluate the recall improvement for an evaluation dataset that was not used in the training process. Deep Memory inference, and by extension this evaluation process, runs on the client.validation_claims, validation_relevances = preprocess_scifact(claims_dataset, dataset_type='validation')recalls = vectorstore.deep_memory.evaluate( queries = validation_claims, relevance = validation_relevances, embedding_function = embedding_function,)We observe that the recall has improved by p to 30%, depending on the k value.recalls---- Evaluating without model ---- Recall@1: 29.5%Recall@3: 45.0%Recall@5: 51.8%Recall@10: 58.1%Recall@50: 77.4%Recall@100: 84.9%---- Evaluating with model ---- Recall@1: 55.1%Recall@3: 68.2%Recall@5: 71.7%Recall@10: 77.9%Recall@50: 90.1%Recall@100: 92.6%


Using Deep Memory in your Application

To use Deep Memory in your applications, specify the deep_memory = True parameter during vector search. If you are using the LangChain integration, you may specify this parameter during Vector Store initialization. Let's try searching embedding using a prompt, with and without Deep Memory.prompt = 'Which diseases are inflammation-related processes'results = vectorstore.search(embedding_data = prompt)results['text']['Inflammation is a fundamental protective response that sometimes goes awry and becomes a major cofactor in the pathogenesis of many chronic human diseases, including cancer.', 'Kidney diseases, including chronic kidney disease (CKD) and acute kidney injury (AKI), are associated with inflammation.', 'BACKGROUND Persistent inflammation has been proposed to contribute to various stages in the pathogenesis of cardiovascular disease.', 'Inflammation accompanies obesity and its comorbidities-type 2 diabetes, non-alcoholic fatty liver disease and atherosclerosis, among others-and may contribute to their pathogenesis.']results_dm = vectorstore.search(embedding_data = prompt, deep_memory = True)results_dm['text']['Kidney diseases, including chronic kidney disease (CKD) and acute kidney injury (AKI), are associated with inflammation.', 'OBJECTIVES Calcific aortic valve (AV) disease is known to be an inflammation-related process.', 'Crohn's disease and ulcerative colitis, the two main types of chronic inflammatory bowel disease, are multifactorial conditions of unknown aetiology.', 'BACKGROUND Two inflammatory disorders, type 1 diabetes and celiac disease, cosegregate in populations, suggesting a common genetic origin.']We observe that there are overlapping results for both search methods, but 50% of the answers differ.Congrats! You just used Deep Memory to improve the accuracy of Vector Search on a specific use-case! 🎉


Deep Lake API Reference

Deep Lake is an open-source database for AI.Getting Started Installation Key Concepts Datasets Creating Datasets Loading Datasets Deleting and Renaming Datasets Copying Datasets Dataset Operations Dataset Visualization Dataset Credentials Dataset Properties Dataset Version Control Dataset Views Vector Store Creating a Deep Lake Vector Store Vector Store Operations Vector Store Properties VectorStore.DeepMemory Creating a Deep Memory Deep Memory Operations Deep Memory Properties Tensors Creating Tensors Deleting and Renaming Tensors Adding and deleting samples Retrieving samples Tensor Properties Info Video features Htypes Image Htype Video Htype Audio Htype Class Label Htype Bounding Box Htype 3D Bounding Box Htype Intrinsics Htype Segmentation Mask Htype Binary Mask Htype COCO Keypoints Htype Point Htype Polygon Htype Nifti Htype Point Cloud Htype Mesh Htype Embedding Htype Sequence htype Link htype Compressions Sample Compression Chunk Compression PyTorch and Tensorflow Support Utility Functions General Functions Making Deep Lake Samples Parallelism Integrations Weights and Biases Logging Dataset Creation Logging Dataset Read MMDetection High-Performance Features Dataloader Sampler Tensor Query Language Random Split Deep Memory API Reference deeplake deeplake.VectorStore deeplake.core deeplake.core.dataset deeplake.core.tensor deeplake.api deeplake.auto deeplake.util deeplake.client.log deeplake.core.transform deeplake.core.vectorstore.deep_memory deeplake.random.seed


Indices and tables

Index Module Index Search Page


Htypes¶

Htype is the class of a tensor: image, bounding box, generic tensor, etc.The htype of a tensor can be specified at its creation Copy to clipboard If not specified, the tensor's htype defaults to 'generic'.Specifying an htype allows for strict settings and error handling, and it is critical for increasing the performance of Deep Lake datasets containing rich data such as images and videos.Supported htypes and their respective defaults are: Htype configs¶ HTYPE DTYPE COMPRESSION generic None None image uint8 Required arg image.rgb uint8 Required arg image.gray uint8 Required arg video uint8 Required arg audio float64 Required arg class_label uint32 None bbox float32 None segment_mask uint32 None binary_mask bool None keypoints_coco int32 None point int32 None polygon float32 None text str None json Any None list List None dicom None dcm point_cloud None las mesh None ply instance_label uint32 None link str None sequence None None Image Htype¶ Sample dimensions: (height, width, # channels) or (height, width).  Images can be stored in Deep Lake as compressed bytes or as raw arrays.  Due to the high compression ratio for most image formats, it is highly recommended to store compressed images using the sample_compression input to the create_tensor method.  Creating an image tensor¶ An image tensor can be created using >>> ds.create_tensor('images', htype='image', sample_compression='jpg') Copy to clipboard OR >>> ds.create_tensor('images', htype='image', chunk_compression='jpg') Copy to clipboard Optional args: dtype: Defaults to uint8.  Supported compressions: >>> [None, 'bmp', 'dib', 'gif', 'ico', 'jpeg', 'jpeg2000', 'pcx', 'png', 'ppm', 'sgi', 'tga', 'tiff', ...  'webp', 'wmf', 'xbm', 'eps', 'fli', 'im', 'msp', 'mpo'] Copy to clipboard Appending image samples¶ Image samples can be of type np.ndarray or Deep Lake Sample which can be created using deeplake.read().  Examples Appending pixel data with array >>> ds.images.append(np.zeros((5, 5, 3), dtype=np.uint8)) Copy to clipboard Appening Deep Lake image sample >>> ds.images.append(deeplake.read('images/0001.jpg')) Copy to clipboard You can append multiple samples at the same time using extend().  >>> ds.images.extend([deeplake.read(f'images/000{i}.jpg') for i in range(10)]) Copy to clipboard Note If the compression format of the input sample does not match the sample_compression of the tensor, Deep Lake will decompress and recompress the image for storage, which may significantly slow down the upload process.  The upload process is fastest when the image compression matches the sample_compression.  image.rgb and image.gray htypes¶ image.rgb and image.gray htypes can be used to force your samples to be of RGB or grayscale type.  i.e., if RGB images are appened to an image.gray tensor, Deep Lake will convert them to grayscale and if grayscale images are appended to an image.rgb tensor, Deep Lake will convert them to RGB format.  image.rgb and image.gray tensors can be created using >>> ds.create_tensor('rgb_images', htype='image.rgb', sample_compression='...') Copy to clipboard >>> ds.create_tensor('gray_images', htype='image.gray', sample_compression='...') Copy to clipboard Video Htype¶ Sample dimensions: (# frames, height, width, # channels) or (# frames, height, width) Creating a video tensor¶ A video tensor can be created using >>> ds.create_tensor('videos', htype='video', sample_compression='mp4') Copy to clipboard Optional args: dtype: Defaults to uint8


Htypes¶

Supported compressions: >>> [None, 'mp4', 'mkv', 'avi'] Copy to clipboard Appending video samples¶ Video samples can be of type np.ndarray or Sample which is returned by deeplake.read() .  Deep Lake does not support compression of raw video frames.  Therefore, array of raw frames can only be appended to tensors with None compression.  Recompression of samples read with deeplake.read is also not supported.  Examples Appending Deep Lake video sample >>> ds.videos.append(deeplake.read('videos/0012.mp4')) Copy to clipboard Extending with multiple videos >>> ds.videos.extend([deeplake.read(f'videos/00{i}.mp4') for i in range(10)]) Copy to clipboard Audio Htype¶ Sample dimensions: (# samples in audio, # channels) or (# samples in audio,) Creating an audio tensor¶ An audio tensor can be created using >>> ds.create_tensor('audios', htype='audio', sample_compression='mp3') Copy to clipboard Optional args: dtype: Defaults to float64.  Supported compressions: >>> [None, 'mp3', 'wav', 'flac'] Copy to clipboard Appending audio samples¶ Audio samples can be of type np.ndarray or Sample which is returned by deeplake.read().  Like videos, Deep Lake does not support compression or recompression of input audio samples.  Thus, samples of type np.ndarray can only be appended to tensors with None compression.  Examples Appending Deep Lake audio sample >>> ds.audios.append(deeplake.read('audios/001.mp3')) Copy to clipboard Extending with Deep Lake audio samples >>> ds.audios.extend([deeplake.read(f'videos/00{i}.mp3') for i in range(10)]) Copy to clipboard Class Label Htype¶ Sample dimensions: (# labels,) Class labels are stored as numerical values in tensors, which are indices of the list tensor.info.class_names.  Creating a class label tensor¶ A class label tensor can be created using >>> classes = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks'] >>> ds.create_tensor('labels', htype='class_label', class_names=classes, chunk_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings.  tensor.info.class_names will be set to this list.  sample_compression or chunk_compression.  dtype: Defaults to uint32.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation.  >>> ds.labels.info.update(class_names = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks']) Copy to clipboard Note If specifying compression, since the number of labels in one sample will be too low, chunk_compression would be the better option to use.  Appending class labels¶ Class labels can be appended as int, str, np.ndarray or list of int or str.  In case of strings, tensor.info.class_names is updated automatically.  Examples Appending index >>> ds.labels.append(0) >>> ds.labels.append(np.zeros((5,), dtype=np.uint32)) Copy to clipboard Extending with list of indices >>> ds.labels.extend([[0, 1, 2], [1, 3]]) Copy to clipboard Appending text labels >>> ds.labels.append(['cars', 'airplanes']) Copy to clipboard Bounding Box Htype¶ Sample dimensions: (# bounding boxes, 4) Bounding boxes have a variety of formats such as YOLO, COCO, Pascal-VOC and others.  In order for bounding boxes to be correctly displayed by the visualizer, the format of the bounding box must be specified in the coords key in tensor meta information mentioned below


Htypes¶

Creating a bbox tensor¶ A bbox tensor can be created using >>> ds.create_tensor('boxes', htype='bbox', coords={'type': 'fractional', 'mode': 'CCWH'}) Copy to clipboard Optional args: coords: A dictionary with keys 'type' and 'mode' .  type: Specifies the units of bounding box coordinates.  'pixel': is in unit of pixels.  'fractional': is in units relative to the width and height of the image, such as in YOLO format.  mode: Specifies the convention for the 4 coordinates 'LTRB': left_x, top_y, right_x, bottom_y 'LTWH': left_x, top_y, width, height 'CCWH': center_x, center_y, width, height dtype: Defaults to float32.  sample_compression or chunk_compression.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation.  >>> ds.labels.info.update(coords = {'type': 'pixel', 'LTRB'}) Copy to clipboard Note If the bounding box format is not specified, the visualizer will assume a YOLO format (fractional + CCWH) if the box coordinates are < 1 on average.  Otherwise, it will assume the COCO format (pixel + LTWH).  Appending bounding boxes¶ Bounding boxes can be appended as np.ndarrays or list or lists of arrays.  Examples Appending one bounding box >>> box array([[462, 123, 238, 98]]) >>> ds.boxes.append(box) Copy to clipboard Appending sample with 3 bounding boxes >>> boxes array([[965, 110, 262, 77], [462, 123, 238, 98], [688, 108, 279, 116]]) >>> boxes.shape (3, 4) >>> ds.boxes.append(boxes) Copy to clipboard Segmentation Mask Htype¶ Sample dimensions: (height, width) Segmentation masks are 2D representations of class labels where the numerical label data is encoded in an array of same shape as the image.  The numerical values are indices of the list tensor.info.class_names.  Creating a segment_mask tensor¶ A segment_mask tensor can be created using >>> classes = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle'] >>> ds.create_tensor('masks', htype='segment_mask', class_names=classes, sample_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings.  tensor.info.class_names will be set to this list.  sample_compression or chunk_compression dtype: Defaults to uint32.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation.  >>> ds.labels.info.update(class_names = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle']) Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.  Appending segmentation masks¶ Segmentation masks can be appended as np.ndarray.  Examples >>> ds.masks.append(np.zeros((512, 512))) Copy to clipboard Note Since each pixel can only be labeled once, segmentation masks are not appropriate for datasets where objects might overlap, or where multiple objects within the same class must be distinguished.  For these use cases, please use htype = 'binary_mask'.  Binary Mask Htype¶ Sample dimensions: (height, width, # objects in a sample) Binary masks are similar to segmentation masks, except that each object is represented by a channel in the mask.  Each channel in the mask encodes values for a single object


Htypes¶

Each point cloud is a list / array of points .  All points in a sample should have the same number of co-ordinates.  Different point clouds can have different number of points.  Creating a point cloud tensor¶ A point cloud tensor can be created using >>> ds.create_tensor('point_clouds', htype='point_cloud', sample_compression='las') Copy to clipboard Optional args: sample_compression Supported compressions: >>> [None, 'las'] Copy to clipboard Appending point clouds¶ Point clouds can be appended as a np.ndarray.  Examples Appending point clouds with numpy arrays >>> import numpy as np >>> point_cloud1 = np.random.randint(0, 10, (5, 3)) >>> ds.point_clouds.append(point_cloud1) >>> point_cloud2 = np.random.randint(0, 10, (15, 3)) >>> ds.point_clouds.append(point_cloud2) >>> ds.point_clouds.shape >>> (2, None, 3) Copy to clipboard Or we can use deeplake.read() method to add samples >>> import deeplake as dp >>> sample = dp.read('example.las') # point cloud with 100 points >>> ds.point_cloud.append(sample) >>> ds.point_cloud.shape >>> (1, 100, 3) Copy to clipboard Mesh Htype¶ Sample dimensions: (# num_points, 3) Mesh samples can be of type np.ndarray or Sample which is returned by deeplake.read().  Each sample in a tensor of mesh htype is a mesh array (3-D object data).  Each mesh is a list / array of points.  Different meshes can have different number of points.  Creating a mesh tensor¶ A mesh tensor can be created using >>> ds.create_tensor('mesh', htype='mesh', sample_compression='ply') Copy to clipboard Optional args: sample_compression Supported compressions: >>> ['ply'] Copy to clipboard Appending meshes¶ Examples Appending a ply file contatining a mesh data to tensor >>> import deeplake as dp >>> sample = dp.read('example.ply') # mesh with 100 points and 200 faces >>> ds.mesh.append(sample) Copy to clipboard >>> ds.mesh.shape >>> (1, 100, 3) Copy to clipboard Sequence htype¶ A special meta htype for tensors where each sample is a sequence.  The items in the sequence are samples of another htype.  It is a wrapper htype that can wrap other htypes like sequence[image], sequence[video], sequence[text], etc.  Examples >>> ds.create_tensor('seq', htype='sequence') >>> ds.seq.append([1, 2, 3]) >>> ds.seq.append([4, 5, 6]) >>> ds.seq.numpy() array([[[1], [2], [3]], [[4], [5], [6]]]) Copy to clipboard >>> ds.create_tensor('image_seq', htype='sequence[image]', sample_compression='jpg') >>> ds.image_seq.append([deeplake.read('img01.jpg'), deeplake.read('img02.jpg')]) Copy to clipboard Link htype¶ Link htype is a special meta htype that allows linking of external data (files) to the dataset, without storing the data in the dataset itself.  Moreover, there can be variations in this htype, such as link[image], link[video], link[audio], etc.  that would enable the activeloop visualizer to correctly display the data.  No data is actually loaded until you try to read the sample from a dataset.  There are a few exceptions to this:- If create_shape_tensor=True was specified during create_tensor of the tensor to which this is being added, the shape of the sample is read.  This is True by default.  If create_sample_info_tensor=True was specified during create_tensor of the tensor to which this is being added, the sample info is read.  This is True by default


Htypes¶

For keypoints that are not present in an image, they can be stored with dummy coordinates of x = 0, y = 0, and v = 0, and the visibility will prevent them from being drawn in the visualizer .  Point Htype¶ Sample dimensions: (# points, 2) in case of 2-D (X, Y) co-ordinates or (# points, 3) in case of 3-D (X, Y, Z) co-ordinates of the point.  Points does not contain a fixed mapping across samples between the point order and real-world objects (i.e., point 0 is an elbow, point 1 is a knee, etc.).  If you require such a mapping, use COCO Keypoints Htype.  Creating a point tensor¶ A point tensor can be created using >>> ds.create_tensor('points', htype='point', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to int32.  Supported compressions: >>> ['lz4'] Copy to clipboard Appending point samples¶ Points can be appended as np.ndarray or list.  Examples Appending 2 2-D points >>> ds.points.append([[0, 1], [1, 3]]) Copy to clipboard Appending 2 3-D points >>> ds.points.append(np.zeros((2, 3))) Copy to clipboard Polygon Htype¶ Sample dimensions: (# polygons, # points per polygon, # co-ordinates per point) Each sample in a tensor of polygon htype is a list of polygons.  Each polygon is a list / array of points.  All points in a sample should have the same number of co-ordinates (eg., cannot mix 2-D points with 3-D points).  Different samples can have different number of polygons.  Different polygons can have different number of points.  Creating a polygon tensor¶ A polygon tensor can be created using >>> ds.create_tensor('polygons', htype='polygon', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to float32.  Supported compressions: >>> ['lz4'] Copy to clipboard Appending polygons¶ Polygons can be appended as a list of list of tuples or np.ndarray.  Examples Appending polygons with 2-D points >>> poly1 = [(1, 2), (2, 3), (3, 4)] >>> poly2 = [(10, 12), (14, 19)] >>> poly3 = [(33, 32), (54, 67), (67, 43), (56, 98)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with 3-D points >>> poly1 = [(10, 2, 9), (12, 3, 8), (12, 10, 4)] >>> poly2 = [(10, 1, 8), (5, 17, 11)] >>> poly3 = [(33, 33, 31), (45, 76, 13), (60, 24, 17), (67, 87, 83)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with numpy arrays >>> import numpy as np >>> sample = np.random.randint(0, 10, (5, 7, 2)) # 5 polygons with 7 points >>> ds.polygons.append(sample) Copy to clipboard >>> import numpy as np >>> poly1 = np.random.randint(0, 10, (5, 2)) >>> poly2 = np.random.randint(0, 10, (8, 2)) >>> poly3 = np.random.randint(0, 10, (3, 2)) >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Point Cloud Htype¶ Sample dimensions: (# num_points, 3) Point cloud samples can be of type np.ndarray or Sample which is returned by deeplake.read()


Creating an image tensor¶

An image tensor can be created using Copy to clipboard OR Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: Copy to clipboard


Htypes¶

A pixel in a mask channel should have a value of 1 if the pixel of the image belongs to this object and 0 otherwise .  The labels corresponding to the channels should be stored in an adjacent tensor of htype class_label, in which the number of labels at a given index is equal to the number of objects (number of channels) in the binary mask.  Creating a binary_mask tensor¶ A binary_mask tensor can be created using >>> ds.create_tensor('masks', htype='binary_mask', sample_compression='lz4') Copy to clipboard Optional args: ref:sample_compression <sample_compression> or chunk_compression dtype: Defaults to bool.  Supported compressions: >>> ['lz4'] Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.  Appending binary masks¶ Binary masks can be appended as np.ndarray.  Examples Appending a binary mask with 5 objects >>> ds.masks.append(np.zeros((512, 512, 5), dtype='bool')) >>> ds.labels.append(['aeroplane', 'aeroplane', 'bottle', 'bottle', 'bird']) Copy to clipboard COCO Keypoints Htype¶ Sample dimensions: (3 x # keypoints, # objects in a sample) COCO keypoints are a convention for storing points of interest in an image.  Each keypoint consists of 3 values: x - coordinate, y - coordinate and v - visibility.  A set of K keypoints of an object is represented as: [x1, y1, v1, x2, y2, v2, …, xk, yk, vk] The visibility v can be one of three values: 0: keypoint not in image.  1: keypoint in image but not visible.  2: keypoint in image and visible.  Creating a keypoints_coco tensor¶ A keypoints_coco tensor can be created using >>> ds.create_tensor('keypoints', htype='keypoints_coco', keypoints=['knee', 'elbow', 'head'], connections=[[0, 1], [1, 2]]) Copy to clipboard Optional args: keypoints: List of strings describing the i th keypoint.  tensor.info.keypoints will be set to this list.  connections: List of strings describing which points should be connected by lines in the visualizer.  sample_compression or chunk_compression dtype: Defaults to int32.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set keypoints and / or connections after tensor creation.  >>> ds.keypoints.info.update(keypoints = ['knee', 'elbow',...]) >>> ds.keypoints.info.update(connections = [[0,1], [2,3], ...]) Copy to clipboard Appending keypoints¶ Keypoints can be appended as np.ndarray or list.  Examples Appending keypoints sample with 3 keypoints and 4 objects >>> ds.keypoints.update(keypoints = ['left ear', 'right ear', 'nose']) >>> ds.keypoints.update(connections = [[0, 2], [1, 2]]) >>> kp_arr array([[465, 398, 684, 469], [178, 363, 177, 177], [ 2, 2, 2, 1], [454, 387, 646, 478], [177, 322, 137, 161], [ 2, 2, 2, 2], [407, 379, 536, 492], [271, 335, 150, 143], [ 2, 1, 2, 2]]) >>> kp_arr.shape (9, 4) >>> ds.keypoints.append(kp_arr) Copy to clipboard Warning In order to correctly use the keypoints and connections metadata, it is critical that all objects in every sample have the same number of K keypoints in the same order


Htypes¶

If verify=True was specified during create_tensor of the tensor to which this is being added, some metadata is read from them to verify the integrity of the link samples .  This is True by default.  If you do not want to verify your links, all three of verify, create_shape_tensor and create_sample_info_tensor have to be set to False.  Examples >>> ds = deeplake.dataset('......') Copy to clipboard Adding credentials to the dataset You can add the names of the credentials you want to use (not needed for http/local urls) >>> ds.add_creds_key('MY_S3_KEY') >>> ds.add_creds_key('GCS_KEY') Copy to clipboard and populate the added names with credentials dictionaries >>> ds.populate_creds('MY_S3_KEY', {}) # add creds here >>> ds.populate_creds('GCS_KEY', {}) # add creds here Copy to clipboard These creds are only present temporarily and will have to be repopulated on every reload.  For datasets connected to Activeloop Platform, you can store your credentials on the platform as Managed Credentials and use them just by adding the keys to your dataset.  For example if you have managed credentials with names 'my_s3_creds', 'my_gcs_creds', you can add them to your dataset using Dataset.add_creds_key without having to populate them.  >>> ds.add_creds_key('my_s3_creds', managed=True) >>> ds.add_creds_key('my_gcs_creds', managed=True) Copy to clipboard Create a link tensor >>> ds.create_tensor('img', htype='link[image]', sample_compression='jpg') Copy to clipboard Populate the tensor with links >>> ds.img.append(deeplake.link('s3://abc/def.jpeg', creds_key='my_s3_key')) >>> ds.img.append(deeplake.link('gcs://ghi/jkl.png', creds_key='GCS_KEY')) >>> ds.img.append(deeplake.link('https://picsum.photos/200/300')) # http path doesn't need creds >>> ds.img.append(deeplake.link('./path/to/cat.jpeg')) # local path doesn't need creds >>> ds.img.append(deeplake.link('s3://abc/def.jpeg')) # this will throw an exception as cloud paths always need creds_key >>> ds.img.append(deeplake.link('s3://abc/def.jpeg', creds_key='ENV')) # this will use creds from environment Copy to clipboard Accessing the data >>> for i in range(5): ...  ds.img[i].numpy() ...  Copy to clipboard Updating a sample >>> ds.img[0] = deeplake.link('./data/cat.jpeg') Copy to clipboard


image.rgb and image.gray htypes¶

image.rgb and image.gray htypes can be used to force your samples to be of RGB or grayscale type. i.e., if RGB images are appened to an image.gray tensor, Deep Lake will convert them to grayscale and if grayscale images are appended to an image.rgb tensor, Deep Lake will convert them to RGB format.image.rgb and image.gray tensors can be created using Copy to clipboard Copy to clipboard


Appending image samples¶

Image samples can be of type np.ndarray or Deep Lake Sample which can be created using deeplake.read(). ExamplesAppending pixel data with array Copy to clipboard Appening Deep Lake image sample Copy to clipboard You can append multiple samples at the same time using extend(). Copy to clipboard NoteIf the compression format of the input sample does not match the sample_compression of the tensor, Deep Lake will decompress and recompress the image for storage, which may significantly slow down the upload process. The upload process is fastest when the image compression matches the sample_compression.


Image Htype¶

Sample dimensions: (height, width, # channels) or (height, width). Images can be stored in Deep Lake as compressed bytes or as raw arrays. Due to the high compression ratio for most image formats, it is highly recommended to store compressed images using the sample_compression input to the create_tensor method. Creating an image tensor¶ An image tensor can be created using >>> ds.create_tensor('images', htype='image', sample_compression='jpg') Copy to clipboard OR >>> ds.create_tensor('images', htype='image', chunk_compression='jpg') Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: >>> [None, 'bmp', 'dib', 'gif', 'ico', 'jpeg', 'jpeg2000', 'pcx', 'png', 'ppm', 'sgi', 'tga', 'tiff', ... 'webp', 'wmf', 'xbm', 'eps', 'fli', 'im', 'msp', 'mpo'] Copy to clipboard Appending image samples¶ Image samples can be of type np.ndarray or Deep Lake Sample which can be created using deeplake.read(). Examples Appending pixel data with array >>> ds.images.append(np.zeros((5, 5, 3), dtype=np.uint8)) Copy to clipboard Appening Deep Lake image sample >>> ds.images.append(deeplake.read('images/0001.jpg')) Copy to clipboard You can append multiple samples at the same time using extend(). >>> ds.images.extend([deeplake.read(f'images/000{i}.jpg') for i in range(10)]) Copy to clipboard Note If the compression format of the input sample does not match the sample_compression of the tensor, Deep Lake will decompress and recompress the image for storage, which may significantly slow down the upload process. The upload process is fastest when the image compression matches the sample_compression. image.rgb and image.gray htypes¶ image.rgb and image.gray htypes can be used to force your samples to be of RGB or grayscale type. i.e., if RGB images are appened to an image.gray tensor, Deep Lake will convert them to grayscale and if grayscale images are appended to an image.rgb tensor, Deep Lake will convert them to RGB format. image.rgb and image.gray tensors can be created using >>> ds.create_tensor('rgb_images', htype='image.rgb', sample_compression='...') Copy to clipboard >>> ds.create_tensor('gray_images', htype='image.gray', sample_compression='...') Copy to clipboard


Appending video samples¶

Video samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Deep Lake does not support compression of raw video frames. Therefore, array of raw frames can only be appended to tensors with None compression. Recompression of samples read with deeplake.read is also not supported. ExamplesAppending Deep Lake video sample Copy to clipboard Extending with multiple videos Copy to clipboard


Video Htype¶

Sample dimensions: (# frames, height, width, # channels) or (# frames, height, width) Creating a video tensor¶ A video tensor can be created using >>> ds.create_tensor('videos', htype='video', sample_compression='mp4') Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: >>> [None, 'mp4', 'mkv', 'avi'] Copy to clipboard Appending video samples¶ Video samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Deep Lake does not support compression of raw video frames. Therefore, array of raw frames can only be appended to tensors with None compression. Recompression of samples read with deeplake.read is also not supported. Examples Appending Deep Lake video sample >>> ds.videos.append(deeplake.read('videos/0012.mp4')) Copy to clipboard Extending with multiple videos >>> ds.videos.extend([deeplake.read(f'videos/00{i}.mp4') for i in range(10)]) Copy to clipboard


Creating an audio tensor¶

An audio tensor can be created using Copy to clipboard Optional args: dtype: Defaults to float64. Supported compressions: Copy to clipboard


Audio Htype¶

Sample dimensions: (# samples in audio, # channels) or (# samples in audio,) Creating an audio tensor¶ An audio tensor can be created using >>> ds.create_tensor('audios', htype='audio', sample_compression='mp3') Copy to clipboard Optional args: dtype: Defaults to float64. Supported compressions: >>> [None, 'mp3', 'wav', 'flac'] Copy to clipboard Appending audio samples¶ Audio samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Like videos, Deep Lake does not support compression or recompression of input audio samples. Thus, samples of type np.ndarray can only be appended to tensors with None compression. Examples Appending Deep Lake audio sample >>> ds.audios.append(deeplake.read('audios/001.mp3')) Copy to clipboard Extending with Deep Lake audio samples >>> ds.audios.extend([deeplake.read(f'videos/00{i}.mp3') for i in range(10)]) Copy to clipboard


Appending audio samples¶

Audio samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Like videos, Deep Lake does not support compression or recompression of input audio samples. Thus, samples of type np.ndarray can only be appended to tensors with None compression. ExamplesAppending Deep Lake audio sample Copy to clipboard Extending with Deep Lake audio samples Copy to clipboard


Class Label Htype¶

Sample dimensions: (# labels,) Class labels are stored as numerical values in tensors, which are indices of the list tensor.info.class_names. Creating a class label tensor¶ A class label tensor can be created using >>> classes = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks'] >>> ds.create_tensor('labels', htype='class_label', class_names=classes, chunk_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression. dtype: Defaults to uint32. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation. >>> ds.labels.info.update(class_names = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks']) Copy to clipboard Note If specifying compression, since the number of labels in one sample will be too low, chunk_compression would be the better option to use. Appending class labels¶ Class labels can be appended as int, str, np.ndarray or list of int or str. In case of strings, tensor.info.class_names is updated automatically. Examples Appending index >>> ds.labels.append(0) >>> ds.labels.append(np.zeros((5,), dtype=np.uint32)) Copy to clipboard Extending with list of indices >>> ds.labels.extend([[0, 1, 2], [1, 3]]) Copy to clipboard Appending text labels >>> ds.labels.append(['cars', 'airplanes']) Copy to clipboard


Creating a class label tensor¶

A class label tensor can be created using Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression. dtype: Defaults to uint32. Supported compressions: Copy to clipboard You can also choose to set the class names after tensor creation. Copy to clipboard NoteIf specifying compression, since the number of labels in one sample will be too low, chunk_compression would be the better option to use.


Bounding Box Htype¶

Sample dimensions: (# bounding boxes, 4) Bounding boxes have a variety of formats such as YOLO, COCO, Pascal-VOC and others. In order for bounding boxes to be correctly displayed by the visualizer, the format of the bounding box must be specified in the coords key in tensor meta information mentioned below. Creating a bbox tensor¶ A bbox tensor can be created using >>> ds.create_tensor('boxes', htype='bbox', coords={'type': 'fractional', 'mode': 'CCWH'}) Copy to clipboard Optional args: coords: A dictionary with keys 'type' and 'mode'. type: Specifies the units of bounding box coordinates. 'pixel': is in unit of pixels. 'fractional': is in units relative to the width and height of the image, such as in YOLO format. mode: Specifies the convention for the 4 coordinates 'LTRB': left_x, top_y, right_x, bottom_y 'LTWH': left_x, top_y, width, height 'CCWH': center_x, center_y, width, height dtype: Defaults to float32. sample_compression or chunk_compression. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation. >>> ds.labels.info.update(coords = {'type': 'pixel', 'LTRB'}) Copy to clipboard Note If the bounding box format is not specified, the visualizer will assume a YOLO format (fractional + CCWH) if the box coordinates are < 1 on average. Otherwise, it will assume the COCO format (pixel + LTWH). Appending bounding boxes¶ Bounding boxes can be appended as np.ndarrays or list or lists of arrays. Examples Appending one bounding box >>> box array([[462, 123, 238, 98]]) >>> ds.boxes.append(box) Copy to clipboard Appending sample with 3 bounding boxes >>> boxes array([[965, 110, 262, 77], [462, 123, 238, 98], [688, 108, 279, 116]]) >>> boxes.shape (3, 4) >>> ds.boxes.append(boxes) Copy to clipboard


Creating a bbox tensor¶

A bbox tensor can be created using Copy to clipboard Optional args: coords: A dictionary with keys 'type' and 'mode'. type: Specifies the units of bounding box coordinates. 'pixel': is in unit of pixels. 'fractional': is in units relative to the width and height of the image, such as in YOLO format. mode: Specifies the convention for the 4 coordinates 'LTRB': left_x, top_y, right_x, bottom_y 'LTWH': left_x, top_y, width, height 'CCWH': center_x, center_y, width, height dtype: Defaults to float32. sample_compression or chunk_compression. Supported compressions: Copy to clipboard You can also choose to set the class names after tensor creation. Copy to clipboard NoteIf the bounding box format is not specified, the visualizer will assume a YOLO format (fractional + CCWH) if the box coordinates are < 1 on average. Otherwise, it will assume the COCO format (pixel + LTWH).


Appending bounding boxes¶

Bounding boxes can be appended as np.ndarrays or list or lists of arrays. ExamplesAppending one bounding box Copy to clipboard Appending sample with 3 bounding boxes Copy to clipboard


Segmentation Mask Htype¶

Sample dimensions: (height, width) Segmentation masks are 2D representations of class labels where the numerical label data is encoded in an array of same shape as the image. The numerical values are indices of the list tensor.info.class_names. Creating a segment_mask tensor¶ A segment_mask tensor can be created using >>> classes = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle'] >>> ds.create_tensor('masks', htype='segment_mask', class_names=classes, sample_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression dtype: Defaults to uint32. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation. >>> ds.labels.info.update(class_names = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle']) Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4. Appending segmentation masks¶ Segmentation masks can be appended as np.ndarray. Examples >>> ds.masks.append(np.zeros((512, 512))) Copy to clipboard Note Since each pixel can only be labeled once, segmentation masks are not appropriate for datasets where objects might overlap, or where multiple objects within the same class must be distinguished. For these use cases, please use htype = 'binary_mask'.


Creating a segment_mask tensor¶

A segment_mask tensor can be created using Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression dtype: Defaults to uint32. Supported compressions: Copy to clipboard You can also choose to set the class names after tensor creation. Copy to clipboard NoteSince segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.


Appending segmentation masks¶

Segmentation masks can be appended as np.ndarray. Examples Copy to clipboard NoteSince each pixel can only be labeled once, segmentation masks are not appropriate for datasets where objects might overlap, or where multiple objects within the same class must be distinguished. For these use cases, please use htype = 'binary_mask'.


Binary Mask Htype¶

Sample dimensions: (height, width, # objects in a sample) Binary masks are similar to segmentation masks, except that each object is represented by a channel in the mask. Each channel in the mask encodes values for a single object. A pixel in a mask channel should have a value of 1 if the pixel of the image belongs to this object and 0 otherwise. The labels corresponding to the channels should be stored in an adjacent tensor of htype class_label, in which the number of labels at a given index is equal to the number of objects (number of channels) in the binary mask. Creating a binary_mask tensor¶ A binary_mask tensor can be created using >>> ds.create_tensor('masks', htype='binary_mask', sample_compression='lz4') Copy to clipboard Optional args: ref:sample_compression <sample_compression> or chunk_compression dtype: Defaults to bool. Supported compressions: >>> ['lz4'] Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4. Appending binary masks¶ Binary masks can be appended as np.ndarray. Examples Appending a binary mask with 5 objects >>> ds.masks.append(np.zeros((512, 512, 5), dtype='bool')) >>> ds.labels.append(['aeroplane', 'aeroplane', 'bottle', 'bottle', 'bird']) Copy to clipboard


Creating a video tensor¶

A video tensor can be created using Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: Copy to clipboard


Creating a binary_mask tensor¶

A binary_mask tensor can be created using Copy to clipboard Optional args: ref:sample_compression <sample_compression> or chunk_compression dtype: Defaults to bool. Supported compressions: Copy to clipboard NoteSince segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.


Appending binary masks¶

Binary masks can be appended as np.ndarray. ExamplesAppending a binary mask with 5 objects Copy to clipboard


Creating a keypoints_coco tensor¶

A keypoints_coco tensor can be created using Copy to clipboard Optional args: keypoints: List of strings describing the i th keypoint. tensor.info.keypoints will be set to this list. connections: List of strings describing which points should be connected by lines in the visualizer. sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: Copy to clipboard You can also choose to set keypoints and / or connections after tensor creation. Copy to clipboard


Appending keypoints¶

Keypoints can be appended as np.ndarray or list. ExamplesAppending keypoints sample with 3 keypoints and 4 objects Copy to clipboard WarningIn order to correctly use the keypoints and connections metadata, it is critical that all objects in every sample have the same number of K keypoints in the same order. For keypoints that are not present in an image, they can be stored with dummy coordinates of x = 0, y = 0, and v = 0, and the visibility will prevent them from being drawn in the visualizer.


COCO Keypoints Htype¶

Sample dimensions: (3 x # keypoints, # objects in a sample) COCO keypoints are a convention for storing points of interest in an image. Each keypoint consists of 3 values: x - coordinate, y - coordinate and v - visibility. A set of K keypoints of an object is represented as:[x1, y1, v1, x2, y2, v2, …, xk, yk, vk]The visibility v can be one of three values: 0: keypoint not in image. 1: keypoint in image but not visible. 2: keypoint in image and visible. Creating a keypoints_coco tensor¶ A keypoints_coco tensor can be created using >>> ds.create_tensor('keypoints', htype='keypoints_coco', keypoints=['knee', 'elbow', 'head'], connections=[[0, 1], [1, 2]]) Copy to clipboard Optional args: keypoints: List of strings describing the i th keypoint. tensor.info.keypoints will be set to this list. connections: List of strings describing which points should be connected by lines in the visualizer. sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set keypoints and / or connections after tensor creation. >>> ds.keypoints.info.update(keypoints = ['knee', 'elbow',...]) >>> ds.keypoints.info.update(connections = [[0,1], [2,3], ...]) Copy to clipboard Appending keypoints¶ Keypoints can be appended as np.ndarray or list. Examples Appending keypoints sample with 3 keypoints and 4 objects >>> ds.keypoints.update(keypoints = ['left ear', 'right ear', 'nose']) >>> ds.keypoints.update(connections = [[0, 2], [1, 2]]) >>> kp_arr array([[465, 398, 684, 469], [178, 363, 177, 177], [ 2, 2, 2, 1], [454, 387, 646, 478], [177, 322, 137, 161], [ 2, 2, 2, 2], [407, 379, 536, 492], [271, 335, 150, 143], [ 2, 1, 2, 2]]) >>> kp_arr.shape (9, 4) >>> ds.keypoints.append(kp_arr) Copy to clipboard Warning In order to correctly use the keypoints and connections metadata, it is critical that all objects in every sample have the same number of K keypoints in the same order. For keypoints that are not present in an image, they can be stored with dummy coordinates of x = 0, y = 0, and v = 0, and the visibility will prevent them from being drawn in the visualizer.


Point Htype¶

Sample dimensions: (# points, 2) in case of 2-D (X, Y) co-ordinates or (# points, 3) in case of 3-D (X, Y, Z) co-ordinates of the point. Points does not contain a fixed mapping across samples between the point order and real-world objects (i.e., point 0 is an elbow, point 1 is a knee, etc.). If you require such a mapping, use COCO Keypoints Htype. Creating a point tensor¶ A point tensor can be created using >>> ds.create_tensor('points', htype='point', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: >>> ['lz4'] Copy to clipboard Appending point samples¶ Points can be appended as np.ndarray or list. Examples Appending 2 2-D points >>> ds.points.append([[0, 1], [1, 3]]) Copy to clipboard Appending 2 3-D points >>> ds.points.append(np.zeros((2, 3))) Copy to clipboard


Creating a point tensor¶

A point tensor can be created using Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: Copy to clipboard


Appending class labels¶

Class labels can be appended as int, str, np.ndarray or list of int or str. In case of strings, tensor.info.class_names is updated automatically. ExamplesAppending index Copy to clipboard Extending with list of indices Copy to clipboard Appending text labels Copy to clipboard


Polygon Htype¶

Sample dimensions: (# polygons, # points per polygon, # co-ordinates per point) Each sample in a tensor of polygon htype is a list of polygons. Each polygon is a list / array of points. All points in a sample should have the same number of co-ordinates (eg., cannot mix 2-D points with 3-D points). Different samples can have different number of polygons. Different polygons can have different number of points. Creating a polygon tensor¶ A polygon tensor can be created using >>> ds.create_tensor('polygons', htype='polygon', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to float32. Supported compressions: >>> ['lz4'] Copy to clipboard Appending polygons¶ Polygons can be appended as a list of list of tuples or np.ndarray. Examples Appending polygons with 2-D points >>> poly1 = [(1, 2), (2, 3), (3, 4)] >>> poly2 = [(10, 12), (14, 19)] >>> poly3 = [(33, 32), (54, 67), (67, 43), (56, 98)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with 3-D points >>> poly1 = [(10, 2, 9), (12, 3, 8), (12, 10, 4)] >>> poly2 = [(10, 1, 8), (5, 17, 11)] >>> poly3 = [(33, 33, 31), (45, 76, 13), (60, 24, 17), (67, 87, 83)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with numpy arrays >>> import numpy as np >>> sample = np.random.randint(0, 10, (5, 7, 2)) # 5 polygons with 7 points >>> ds.polygons.append(sample) Copy to clipboard >>> import numpy as np >>> poly1 = np.random.randint(0, 10, (5, 2)) >>> poly2 = np.random.randint(0, 10, (8, 2)) >>> poly3 = np.random.randint(0, 10, (3, 2)) >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard


Appending point samples¶

Points can be appended as np.ndarray or list. ExamplesAppending 2 2-D points Copy to clipboard Appending 2 3-D points Copy to clipboard


Appending polygons¶

Polygons can be appended as a list of list of tuples or np.ndarray. ExamplesAppending polygons with 2-D points Copy to clipboard Appending polygons with 3-D points Copy to clipboard Appending polygons with numpy arrays Copy to clipboard Copy to clipboard


Point Cloud Htype¶

Sample dimensions: (# num_points, 3) Point cloud samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Each point cloud is a list / array of points. All points in a sample should have the same number of co-ordinates. Different point clouds can have different number of points. Creating a point cloud tensor¶ A point cloud tensor can be created using >>> ds.create_tensor('point_clouds', htype='point_cloud', sample_compression='las') Copy to clipboard Optional args: sample_compression Supported compressions: >>> [None, 'las'] Copy to clipboard Appending point clouds¶ Point clouds can be appended as a np.ndarray. Examples Appending point clouds with numpy arrays >>> import numpy as np >>> point_cloud1 = np.random.randint(0, 10, (5, 3)) >>> ds.point_clouds.append(point_cloud1) >>> point_cloud2 = np.random.randint(0, 10, (15, 3)) >>> ds.point_clouds.append(point_cloud2) >>> ds.point_clouds.shape >>> (2, None, 3) Copy to clipboard Or we can use deeplake.read() method to add samples >>> import deeplake as dp >>> sample = dp.read('example.las') # point cloud with 100 points >>> ds.point_cloud.append(sample) >>> ds.point_cloud.shape >>> (1, 100, 3) Copy to clipboard


Appending point clouds¶

Point clouds can be appended as a np.ndarray. ExamplesAppending point clouds with numpy arrays Copy to clipboard Or we can use deeplake.read() method to add samples Copy to clipboard


Creating a point cloud tensor¶

A point cloud tensor can be created using Copy to clipboard Optional args: sample_compression Supported compressions: Copy to clipboard


Creating a polygon tensor¶

A polygon tensor can be created using Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to float32. Supported compressions: Copy to clipboard


Mesh Htype¶

Sample dimensions: (# num_points, 3) Mesh samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Each sample in a tensor of mesh htype is a mesh array (3-D object data). Each mesh is a list / array of points. Different meshes can have different number of points. Creating a mesh tensor¶ A mesh tensor can be created using >>> ds.create_tensor('mesh', htype='mesh', sample_compression='ply') Copy to clipboard Optional args: sample_compression Supported compressions: >>> ['ply'] Copy to clipboard Appending meshes¶ Examples Appending a ply file contatining a mesh data to tensor >>> import deeplake as dp >>> sample = dp.read('example.ply') # mesh with 100 points and 200 faces >>> ds.mesh.append(sample) Copy to clipboard >>> ds.mesh.shape >>> (1, 100, 3) Copy to clipboard


Appending meshes¶

ExamplesAppending a ply file contatining a mesh data to tensor Copy to clipboard Copy to clipboard


Sequence htype¶

A special meta htype for tensors where each sample is a sequence. The items in the sequence are samples of another htype. It is a wrapper htype that can wrap other htypes like sequence[image], sequence[video], sequence[text], etc. Examples Copy to clipboard Copy to clipboard


Link htype¶

Link htype is a special meta htype that allows linking of external data (files) to the dataset, without storing the data in the dataset itself. Moreover, there can be variations in this htype, such as link[image], link[video], link[audio], etc. that would enable the activeloop visualizer to correctly display the data. No data is actually loaded until you try to read the sample from a dataset. There are a few exceptions to this:- If create_shape_tensor=True was specified during create_tensor of the tensor to which this is being added, the shape of the sample is read. This is True by default. If create_sample_info_tensor=True was specified during create_tensor of the tensor to which this is being added, the sample info is read. This is True by default. If verify=True was specified during create_tensor of the tensor to which this is being added, some metadata is read from them to verify the integrity of the link samples. This is True by default. If you do not want to verify your links, all three of verify, create_shape_tensor and create_sample_info_tensor have to be set to False. Examples Copy to clipboard Adding credentials to the datasetYou can add the names of the credentials you want to use (not needed for http/local urls) Copy to clipboard and populate the added names with credentials dictionaries Copy to clipboard These creds are only present temporarily and will have to be repopulated on every reload.For datasets connected to Activeloop Platform, you can store your credentials on the platform as Managed Credentials and use them just by adding the keys to your dataset. For example if you have managed credentials with names 'my_s3_creds', 'my_gcs_creds', you can add them to your dataset using Dataset.add_creds_key without having to populate them. Copy to clipboard Create a link tensor Copy to clipboard Populate the tensor with links Copy to clipboard Accessing the data Copy to clipboard Updating a sample Copy to clipboard


Creating a mesh tensor¶

A mesh tensor can be created using Copy to clipboard Optional args: sample_compression Supported compressions: Copy to clipboard


Htypes¶

Htype is the class of a tensor: image, bounding box, generic tensor, etc.The htype of a tensor can be specified at its creation Copy to clipboard If not specified, the tensor's htype defaults to 'generic'.Specifying an htype allows for strict settings and error handling, and it is critical for increasing the performance of Deep Lake datasets containing rich data such as images and videos.Supported htypes and their respective defaults are: Htype configs¶ HTYPE DTYPE COMPRESSION generic None None image uint8 Required arg image.rgb uint8 Required arg image.gray uint8 Required arg video uint8 Required arg audio float64 Required arg class_label uint32 None bbox float32 None segment_mask uint32 None binary_mask bool None keypoints_coco int32 None point int32 None polygon float32 None text str None json Any None list List None dicom None dcm point_cloud None las mesh None ply instance_label uint32 None link str None sequence None None Image Htype¶ Sample dimensions: (height, width, # channels) or (height, width).  Images can be stored in Deep Lake as compressed bytes or as raw arrays.  Due to the high compression ratio for most image formats, it is highly recommended to store compressed images using the sample_compression input to the create_tensor method.  Creating an image tensor¶ An image tensor can be created using >>> ds.create_tensor('images', htype='image', sample_compression='jpg') Copy to clipboard OR >>> ds.create_tensor('images', htype='image', chunk_compression='jpg') Copy to clipboard Optional args: dtype: Defaults to uint8.  Supported compressions: >>> [None, 'bmp', 'dib', 'gif', 'ico', 'jpeg', 'jpeg2000', 'pcx', 'png', 'ppm', 'sgi', 'tga', 'tiff', ...  'webp', 'wmf', 'xbm', 'eps', 'fli', 'im', 'msp', 'mpo'] Copy to clipboard Appending image samples¶ Image samples can be of type np.ndarray or Deep Lake Sample which can be created using deeplake.read().  Examples Appending pixel data with array >>> ds.images.append(np.zeros((5, 5, 3), dtype=np.uint8)) Copy to clipboard Appening Deep Lake image sample >>> ds.images.append(deeplake.read('images/0001.jpg')) Copy to clipboard You can append multiple samples at the same time using extend().  >>> ds.images.extend([deeplake.read(f'images/000{i}.jpg') for i in range(10)]) Copy to clipboard Note If the compression format of the input sample does not match the sample_compression of the tensor, Deep Lake will decompress and recompress the image for storage, which may significantly slow down the upload process.  The upload process is fastest when the image compression matches the sample_compression.  image.rgb and image.gray htypes¶ image.rgb and image.gray htypes can be used to force your samples to be of RGB or grayscale type.  i.e., if RGB images are appened to an image.gray tensor, Deep Lake will convert them to grayscale and if grayscale images are appended to an image.rgb tensor, Deep Lake will convert them to RGB format.  image.rgb and image.gray tensors can be created using >>> ds.create_tensor('rgb_images', htype='image.rgb', sample_compression='...') Copy to clipboard >>> ds.create_tensor('gray_images', htype='image.gray', sample_compression='...') Copy to clipboard Video Htype¶ Sample dimensions: (# frames, height, width, # channels) or (# frames, height, width) Creating a video tensor¶ A video tensor can be created using >>> ds.create_tensor('videos', htype='video', sample_compression='mp4') Copy to clipboard Optional args: dtype: Defaults to uint8


Htypes¶

Creating a bbox tensor¶ A bbox tensor can be created using >>> ds.create_tensor('boxes', htype='bbox', coords={'type': 'fractional', 'mode': 'CCWH'}) Copy to clipboard Optional args: coords: A dictionary with keys 'type' and 'mode' .  type: Specifies the units of bounding box coordinates.  'pixel': is in unit of pixels.  'fractional': is in units relative to the width and height of the image, such as in YOLO format.  mode: Specifies the convention for the 4 coordinates 'LTRB': left_x, top_y, right_x, bottom_y 'LTWH': left_x, top_y, width, height 'CCWH': center_x, center_y, width, height dtype: Defaults to float32.  sample_compression or chunk_compression.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation.  >>> ds.labels.info.update(coords = {'type': 'pixel', 'LTRB'}) Copy to clipboard Note If the bounding box format is not specified, the visualizer will assume a YOLO format (fractional + CCWH) if the box coordinates are < 1 on average.  Otherwise, it will assume the COCO format (pixel + LTWH).  Appending bounding boxes¶ Bounding boxes can be appended as np.ndarrays or list or lists of arrays.  Examples Appending one bounding box >>> box array([[462, 123, 238, 98]]) >>> ds.boxes.append(box) Copy to clipboard Appending sample with 3 bounding boxes >>> boxes array([[965, 110, 262, 77], [462, 123, 238, 98], [688, 108, 279, 116]]) >>> boxes.shape (3, 4) >>> ds.boxes.append(boxes) Copy to clipboard Segmentation Mask Htype¶ Sample dimensions: (height, width) Segmentation masks are 2D representations of class labels where the numerical label data is encoded in an array of same shape as the image.  The numerical values are indices of the list tensor.info.class_names.  Creating a segment_mask tensor¶ A segment_mask tensor can be created using >>> classes = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle'] >>> ds.create_tensor('masks', htype='segment_mask', class_names=classes, sample_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings.  tensor.info.class_names will be set to this list.  sample_compression or chunk_compression dtype: Defaults to uint32.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation.  >>> ds.labels.info.update(class_names = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle']) Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.  Appending segmentation masks¶ Segmentation masks can be appended as np.ndarray.  Examples >>> ds.masks.append(np.zeros((512, 512))) Copy to clipboard Note Since each pixel can only be labeled once, segmentation masks are not appropriate for datasets where objects might overlap, or where multiple objects within the same class must be distinguished.  For these use cases, please use htype = 'binary_mask'.  Binary Mask Htype¶ Sample dimensions: (height, width, # objects in a sample) Binary masks are similar to segmentation masks, except that each object is represented by a channel in the mask.  Each channel in the mask encodes values for a single object


Htypes¶

Supported compressions: >>> [None, 'mp4', 'mkv', 'avi'] Copy to clipboard Appending video samples¶ Video samples can be of type np.ndarray or Sample which is returned by deeplake.read() .  Deep Lake does not support compression of raw video frames.  Therefore, array of raw frames can only be appended to tensors with None compression.  Recompression of samples read with deeplake.read is also not supported.  Examples Appending Deep Lake video sample >>> ds.videos.append(deeplake.read('videos/0012.mp4')) Copy to clipboard Extending with multiple videos >>> ds.videos.extend([deeplake.read(f'videos/00{i}.mp4') for i in range(10)]) Copy to clipboard Audio Htype¶ Sample dimensions: (# samples in audio, # channels) or (# samples in audio,) Creating an audio tensor¶ An audio tensor can be created using >>> ds.create_tensor('audios', htype='audio', sample_compression='mp3') Copy to clipboard Optional args: dtype: Defaults to float64.  Supported compressions: >>> [None, 'mp3', 'wav', 'flac'] Copy to clipboard Appending audio samples¶ Audio samples can be of type np.ndarray or Sample which is returned by deeplake.read().  Like videos, Deep Lake does not support compression or recompression of input audio samples.  Thus, samples of type np.ndarray can only be appended to tensors with None compression.  Examples Appending Deep Lake audio sample >>> ds.audios.append(deeplake.read('audios/001.mp3')) Copy to clipboard Extending with Deep Lake audio samples >>> ds.audios.extend([deeplake.read(f'videos/00{i}.mp3') for i in range(10)]) Copy to clipboard Class Label Htype¶ Sample dimensions: (# labels,) Class labels are stored as numerical values in tensors, which are indices of the list tensor.info.class_names.  Creating a class label tensor¶ A class label tensor can be created using >>> classes = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks'] >>> ds.create_tensor('labels', htype='class_label', class_names=classes, chunk_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings.  tensor.info.class_names will be set to this list.  sample_compression or chunk_compression.  dtype: Defaults to uint32.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation.  >>> ds.labels.info.update(class_names = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks']) Copy to clipboard Note If specifying compression, since the number of labels in one sample will be too low, chunk_compression would be the better option to use.  Appending class labels¶ Class labels can be appended as int, str, np.ndarray or list of int or str.  In case of strings, tensor.info.class_names is updated automatically.  Examples Appending index >>> ds.labels.append(0) >>> ds.labels.append(np.zeros((5,), dtype=np.uint32)) Copy to clipboard Extending with list of indices >>> ds.labels.extend([[0, 1, 2], [1, 3]]) Copy to clipboard Appending text labels >>> ds.labels.append(['cars', 'airplanes']) Copy to clipboard Bounding Box Htype¶ Sample dimensions: (# bounding boxes, 4) Bounding boxes have a variety of formats such as YOLO, COCO, Pascal-VOC and others.  In order for bounding boxes to be correctly displayed by the visualizer, the format of the bounding box must be specified in the coords key in tensor meta information mentioned below


Htypes¶

A pixel in a mask channel should have a value of 1 if the pixel of the image belongs to this object and 0 otherwise .  The labels corresponding to the channels should be stored in an adjacent tensor of htype class_label, in which the number of labels at a given index is equal to the number of objects (number of channels) in the binary mask.  Creating a binary_mask tensor¶ A binary_mask tensor can be created using >>> ds.create_tensor('masks', htype='binary_mask', sample_compression='lz4') Copy to clipboard Optional args: ref:sample_compression <sample_compression> or chunk_compression dtype: Defaults to bool.  Supported compressions: >>> ['lz4'] Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.  Appending binary masks¶ Binary masks can be appended as np.ndarray.  Examples Appending a binary mask with 5 objects >>> ds.masks.append(np.zeros((512, 512, 5), dtype='bool')) >>> ds.labels.append(['aeroplane', 'aeroplane', 'bottle', 'bottle', 'bird']) Copy to clipboard COCO Keypoints Htype¶ Sample dimensions: (3 x # keypoints, # objects in a sample) COCO keypoints are a convention for storing points of interest in an image.  Each keypoint consists of 3 values: x - coordinate, y - coordinate and v - visibility.  A set of K keypoints of an object is represented as: [x1, y1, v1, x2, y2, v2, …, xk, yk, vk] The visibility v can be one of three values: 0: keypoint not in image.  1: keypoint in image but not visible.  2: keypoint in image and visible.  Creating a keypoints_coco tensor¶ A keypoints_coco tensor can be created using >>> ds.create_tensor('keypoints', htype='keypoints_coco', keypoints=['knee', 'elbow', 'head'], connections=[[0, 1], [1, 2]]) Copy to clipboard Optional args: keypoints: List of strings describing the i th keypoint.  tensor.info.keypoints will be set to this list.  connections: List of strings describing which points should be connected by lines in the visualizer.  sample_compression or chunk_compression dtype: Defaults to int32.  Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set keypoints and / or connections after tensor creation.  >>> ds.keypoints.info.update(keypoints = ['knee', 'elbow',...]) >>> ds.keypoints.info.update(connections = [[0,1], [2,3], ...]) Copy to clipboard Appending keypoints¶ Keypoints can be appended as np.ndarray or list.  Examples Appending keypoints sample with 3 keypoints and 4 objects >>> ds.keypoints.update(keypoints = ['left ear', 'right ear', 'nose']) >>> ds.keypoints.update(connections = [[0, 2], [1, 2]]) >>> kp_arr array([[465, 398, 684, 469], [178, 363, 177, 177], [ 2, 2, 2, 1], [454, 387, 646, 478], [177, 322, 137, 161], [ 2, 2, 2, 2], [407, 379, 536, 492], [271, 335, 150, 143], [ 2, 1, 2, 2]]) >>> kp_arr.shape (9, 4) >>> ds.keypoints.append(kp_arr) Copy to clipboard Warning In order to correctly use the keypoints and connections metadata, it is critical that all objects in every sample have the same number of K keypoints in the same order


Htypes¶

Each point cloud is a list / array of points .  All points in a sample should have the same number of co-ordinates.  Different point clouds can have different number of points.  Creating a point cloud tensor¶ A point cloud tensor can be created using >>> ds.create_tensor('point_clouds', htype='point_cloud', sample_compression='las') Copy to clipboard Optional args: sample_compression Supported compressions: >>> [None, 'las'] Copy to clipboard Appending point clouds¶ Point clouds can be appended as a np.ndarray.  Examples Appending point clouds with numpy arrays >>> import numpy as np >>> point_cloud1 = np.random.randint(0, 10, (5, 3)) >>> ds.point_clouds.append(point_cloud1) >>> point_cloud2 = np.random.randint(0, 10, (15, 3)) >>> ds.point_clouds.append(point_cloud2) >>> ds.point_clouds.shape >>> (2, None, 3) Copy to clipboard Or we can use deeplake.read() method to add samples >>> import deeplake as dp >>> sample = dp.read('example.las') # point cloud with 100 points >>> ds.point_cloud.append(sample) >>> ds.point_cloud.shape >>> (1, 100, 3) Copy to clipboard Mesh Htype¶ Sample dimensions: (# num_points, 3) Mesh samples can be of type np.ndarray or Sample which is returned by deeplake.read().  Each sample in a tensor of mesh htype is a mesh array (3-D object data).  Each mesh is a list / array of points.  Different meshes can have different number of points.  Creating a mesh tensor¶ A mesh tensor can be created using >>> ds.create_tensor('mesh', htype='mesh', sample_compression='ply') Copy to clipboard Optional args: sample_compression Supported compressions: >>> ['ply'] Copy to clipboard Appending meshes¶ Examples Appending a ply file contatining a mesh data to tensor >>> import deeplake as dp >>> sample = dp.read('example.ply') # mesh with 100 points and 200 faces >>> ds.mesh.append(sample) Copy to clipboard >>> ds.mesh.shape >>> (1, 100, 3) Copy to clipboard Sequence htype¶ A special meta htype for tensors where each sample is a sequence.  The items in the sequence are samples of another htype.  It is a wrapper htype that can wrap other htypes like sequence[image], sequence[video], sequence[text], etc.  Examples >>> ds.create_tensor('seq', htype='sequence') >>> ds.seq.append([1, 2, 3]) >>> ds.seq.append([4, 5, 6]) >>> ds.seq.numpy() array([[[1], [2], [3]], [[4], [5], [6]]]) Copy to clipboard >>> ds.create_tensor('image_seq', htype='sequence[image]', sample_compression='jpg') >>> ds.image_seq.append([deeplake.read('img01.jpg'), deeplake.read('img02.jpg')]) Copy to clipboard Link htype¶ Link htype is a special meta htype that allows linking of external data (files) to the dataset, without storing the data in the dataset itself.  Moreover, there can be variations in this htype, such as link[image], link[video], link[audio], etc.  that would enable the activeloop visualizer to correctly display the data.  No data is actually loaded until you try to read the sample from a dataset.  There are a few exceptions to this:- If create_shape_tensor=True was specified during create_tensor of the tensor to which this is being added, the shape of the sample is read.  This is True by default.  If create_sample_info_tensor=True was specified during create_tensor of the tensor to which this is being added, the sample info is read.  This is True by default


Image Htype¶

Sample dimensions: (height, width, # channels) or (height, width). Images can be stored in Deep Lake as compressed bytes or as raw arrays. Due to the high compression ratio for most image formats, it is highly recommended to store compressed images using the sample_compression input to the create_tensor method. Creating an image tensor¶ An image tensor can be created using >>> ds.create_tensor('images', htype='image', sample_compression='jpg') Copy to clipboard OR >>> ds.create_tensor('images', htype='image', chunk_compression='jpg') Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: >>> [None, 'bmp', 'dib', 'gif', 'ico', 'jpeg', 'jpeg2000', 'pcx', 'png', 'ppm', 'sgi', 'tga', 'tiff', ... 'webp', 'wmf', 'xbm', 'eps', 'fli', 'im', 'msp', 'mpo'] Copy to clipboard Appending image samples¶ Image samples can be of type np.ndarray or Deep Lake Sample which can be created using deeplake.read(). Examples Appending pixel data with array >>> ds.images.append(np.zeros((5, 5, 3), dtype=np.uint8)) Copy to clipboard Appening Deep Lake image sample >>> ds.images.append(deeplake.read('images/0001.jpg')) Copy to clipboard You can append multiple samples at the same time using extend(). >>> ds.images.extend([deeplake.read(f'images/000{i}.jpg') for i in range(10)]) Copy to clipboard Note If the compression format of the input sample does not match the sample_compression of the tensor, Deep Lake will decompress and recompress the image for storage, which may significantly slow down the upload process. The upload process is fastest when the image compression matches the sample_compression. image.rgb and image.gray htypes¶ image.rgb and image.gray htypes can be used to force your samples to be of RGB or grayscale type. i.e., if RGB images are appened to an image.gray tensor, Deep Lake will convert them to grayscale and if grayscale images are appended to an image.rgb tensor, Deep Lake will convert them to RGB format. image.rgb and image.gray tensors can be created using >>> ds.create_tensor('rgb_images', htype='image.rgb', sample_compression='...') Copy to clipboard >>> ds.create_tensor('gray_images', htype='image.gray', sample_compression='...') Copy to clipboard


Htypes¶

For keypoints that are not present in an image, they can be stored with dummy coordinates of x = 0, y = 0, and v = 0, and the visibility will prevent them from being drawn in the visualizer .  Point Htype¶ Sample dimensions: (# points, 2) in case of 2-D (X, Y) co-ordinates or (# points, 3) in case of 3-D (X, Y, Z) co-ordinates of the point.  Points does not contain a fixed mapping across samples between the point order and real-world objects (i.e., point 0 is an elbow, point 1 is a knee, etc.).  If you require such a mapping, use COCO Keypoints Htype.  Creating a point tensor¶ A point tensor can be created using >>> ds.create_tensor('points', htype='point', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to int32.  Supported compressions: >>> ['lz4'] Copy to clipboard Appending point samples¶ Points can be appended as np.ndarray or list.  Examples Appending 2 2-D points >>> ds.points.append([[0, 1], [1, 3]]) Copy to clipboard Appending 2 3-D points >>> ds.points.append(np.zeros((2, 3))) Copy to clipboard Polygon Htype¶ Sample dimensions: (# polygons, # points per polygon, # co-ordinates per point) Each sample in a tensor of polygon htype is a list of polygons.  Each polygon is a list / array of points.  All points in a sample should have the same number of co-ordinates (eg., cannot mix 2-D points with 3-D points).  Different samples can have different number of polygons.  Different polygons can have different number of points.  Creating a polygon tensor¶ A polygon tensor can be created using >>> ds.create_tensor('polygons', htype='polygon', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to float32.  Supported compressions: >>> ['lz4'] Copy to clipboard Appending polygons¶ Polygons can be appended as a list of list of tuples or np.ndarray.  Examples Appending polygons with 2-D points >>> poly1 = [(1, 2), (2, 3), (3, 4)] >>> poly2 = [(10, 12), (14, 19)] >>> poly3 = [(33, 32), (54, 67), (67, 43), (56, 98)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with 3-D points >>> poly1 = [(10, 2, 9), (12, 3, 8), (12, 10, 4)] >>> poly2 = [(10, 1, 8), (5, 17, 11)] >>> poly3 = [(33, 33, 31), (45, 76, 13), (60, 24, 17), (67, 87, 83)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with numpy arrays >>> import numpy as np >>> sample = np.random.randint(0, 10, (5, 7, 2)) # 5 polygons with 7 points >>> ds.polygons.append(sample) Copy to clipboard >>> import numpy as np >>> poly1 = np.random.randint(0, 10, (5, 2)) >>> poly2 = np.random.randint(0, 10, (8, 2)) >>> poly3 = np.random.randint(0, 10, (3, 2)) >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Point Cloud Htype¶ Sample dimensions: (# num_points, 3) Point cloud samples can be of type np.ndarray or Sample which is returned by deeplake.read()


Htypes¶

If verify=True was specified during create_tensor of the tensor to which this is being added, some metadata is read from them to verify the integrity of the link samples .  This is True by default.  If you do not want to verify your links, all three of verify, create_shape_tensor and create_sample_info_tensor have to be set to False.  Examples >>> ds = deeplake.dataset('......') Copy to clipboard Adding credentials to the dataset You can add the names of the credentials you want to use (not needed for http/local urls) >>> ds.add_creds_key('MY_S3_KEY') >>> ds.add_creds_key('GCS_KEY') Copy to clipboard and populate the added names with credentials dictionaries >>> ds.populate_creds('MY_S3_KEY', {}) # add creds here >>> ds.populate_creds('GCS_KEY', {}) # add creds here Copy to clipboard These creds are only present temporarily and will have to be repopulated on every reload.  For datasets connected to Activeloop Platform, you can store your credentials on the platform as Managed Credentials and use them just by adding the keys to your dataset.  For example if you have managed credentials with names 'my_s3_creds', 'my_gcs_creds', you can add them to your dataset using Dataset.add_creds_key without having to populate them.  >>> ds.add_creds_key('my_s3_creds', managed=True) >>> ds.add_creds_key('my_gcs_creds', managed=True) Copy to clipboard Create a link tensor >>> ds.create_tensor('img', htype='link[image]', sample_compression='jpg') Copy to clipboard Populate the tensor with links >>> ds.img.append(deeplake.link('s3://abc/def.jpeg', creds_key='my_s3_key')) >>> ds.img.append(deeplake.link('gcs://ghi/jkl.png', creds_key='GCS_KEY')) >>> ds.img.append(deeplake.link('https://picsum.photos/200/300')) # http path doesn't need creds >>> ds.img.append(deeplake.link('./path/to/cat.jpeg')) # local path doesn't need creds >>> ds.img.append(deeplake.link('s3://abc/def.jpeg')) # this will throw an exception as cloud paths always need creds_key >>> ds.img.append(deeplake.link('s3://abc/def.jpeg', creds_key='ENV')) # this will use creds from environment Copy to clipboard Accessing the data >>> for i in range(5): ...  ds.img[i].numpy() ...  Copy to clipboard Updating a sample >>> ds.img[0] = deeplake.link('./data/cat.jpeg') Copy to clipboard


Appending image samples¶

Image samples can be of type np.ndarray or Deep Lake Sample which can be created using deeplake.read(). ExamplesAppending pixel data with array Copy to clipboard Appening Deep Lake image sample Copy to clipboard You can append multiple samples at the same time using extend(). Copy to clipboard NoteIf the compression format of the input sample does not match the sample_compression of the tensor, Deep Lake will decompress and recompress the image for storage, which may significantly slow down the upload process. The upload process is fastest when the image compression matches the sample_compression.


Creating an image tensor¶

An image tensor can be created using Copy to clipboard OR Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: Copy to clipboard


Creating a video tensor¶

A video tensor can be created using Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: Copy to clipboard


Appending video samples¶

Video samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Deep Lake does not support compression of raw video frames. Therefore, array of raw frames can only be appended to tensors with None compression. Recompression of samples read with deeplake.read is also not supported. ExamplesAppending Deep Lake video sample Copy to clipboard Extending with multiple videos Copy to clipboard


Audio Htype¶

Sample dimensions: (# samples in audio, # channels) or (# samples in audio,) Creating an audio tensor¶ An audio tensor can be created using >>> ds.create_tensor('audios', htype='audio', sample_compression='mp3') Copy to clipboard Optional args: dtype: Defaults to float64. Supported compressions: >>> [None, 'mp3', 'wav', 'flac'] Copy to clipboard Appending audio samples¶ Audio samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Like videos, Deep Lake does not support compression or recompression of input audio samples. Thus, samples of type np.ndarray can only be appended to tensors with None compression. Examples Appending Deep Lake audio sample >>> ds.audios.append(deeplake.read('audios/001.mp3')) Copy to clipboard Extending with Deep Lake audio samples >>> ds.audios.extend([deeplake.read(f'videos/00{i}.mp3') for i in range(10)]) Copy to clipboard


Creating an audio tensor¶

An audio tensor can be created using Copy to clipboard Optional args: dtype: Defaults to float64. Supported compressions: Copy to clipboard


Appending audio samples¶

Audio samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Like videos, Deep Lake does not support compression or recompression of input audio samples. Thus, samples of type np.ndarray can only be appended to tensors with None compression. ExamplesAppending Deep Lake audio sample Copy to clipboard Extending with Deep Lake audio samples Copy to clipboard


Class Label Htype¶

Sample dimensions: (# labels,) Class labels are stored as numerical values in tensors, which are indices of the list tensor.info.class_names. Creating a class label tensor¶ A class label tensor can be created using >>> classes = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks'] >>> ds.create_tensor('labels', htype='class_label', class_names=classes, chunk_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression. dtype: Defaults to uint32. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation. >>> ds.labels.info.update(class_names = ['airplanes', 'cars', 'birds', 'cats', 'deer', 'dogs', 'frogs', 'horses', 'ships', 'trucks']) Copy to clipboard Note If specifying compression, since the number of labels in one sample will be too low, chunk_compression would be the better option to use. Appending class labels¶ Class labels can be appended as int, str, np.ndarray or list of int or str. In case of strings, tensor.info.class_names is updated automatically. Examples Appending index >>> ds.labels.append(0) >>> ds.labels.append(np.zeros((5,), dtype=np.uint32)) Copy to clipboard Extending with list of indices >>> ds.labels.extend([[0, 1, 2], [1, 3]]) Copy to clipboard Appending text labels >>> ds.labels.append(['cars', 'airplanes']) Copy to clipboard


image.rgb and image.gray htypes¶

image.rgb and image.gray htypes can be used to force your samples to be of RGB or grayscale type. i.e., if RGB images are appened to an image.gray tensor, Deep Lake will convert them to grayscale and if grayscale images are appended to an image.rgb tensor, Deep Lake will convert them to RGB format.image.rgb and image.gray tensors can be created using Copy to clipboard Copy to clipboard


Creating a class label tensor¶

A class label tensor can be created using Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression. dtype: Defaults to uint32. Supported compressions: Copy to clipboard You can also choose to set the class names after tensor creation. Copy to clipboard NoteIf specifying compression, since the number of labels in one sample will be too low, chunk_compression would be the better option to use.


Video Htype¶

Sample dimensions: (# frames, height, width, # channels) or (# frames, height, width) Creating a video tensor¶ A video tensor can be created using >>> ds.create_tensor('videos', htype='video', sample_compression='mp4') Copy to clipboard Optional args: dtype: Defaults to uint8. Supported compressions: >>> [None, 'mp4', 'mkv', 'avi'] Copy to clipboard Appending video samples¶ Video samples can be of type np.ndarray or Sample which is returned by deeplake.read(). Deep Lake does not support compression of raw video frames. Therefore, array of raw frames can only be appended to tensors with None compression. Recompression of samples read with deeplake.read is also not supported. Examples Appending Deep Lake video sample >>> ds.videos.append(deeplake.read('videos/0012.mp4')) Copy to clipboard Extending with multiple videos >>> ds.videos.extend([deeplake.read(f'videos/00{i}.mp4') for i in range(10)]) Copy to clipboard


Bounding Box Htype¶

Sample dimensions: (# bounding boxes, 4) Bounding boxes have a variety of formats such as YOLO, COCO, Pascal-VOC and others. In order for bounding boxes to be correctly displayed by the visualizer, the format of the bounding box must be specified in the coords key in tensor meta information mentioned below. Creating a bbox tensor¶ A bbox tensor can be created using >>> ds.create_tensor('boxes', htype='bbox', coords={'type': 'fractional', 'mode': 'CCWH'}) Copy to clipboard Optional args: coords: A dictionary with keys 'type' and 'mode'. type: Specifies the units of bounding box coordinates. 'pixel': is in unit of pixels. 'fractional': is in units relative to the width and height of the image, such as in YOLO format. mode: Specifies the convention for the 4 coordinates 'LTRB': left_x, top_y, right_x, bottom_y 'LTWH': left_x, top_y, width, height 'CCWH': center_x, center_y, width, height dtype: Defaults to float32. sample_compression or chunk_compression. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation. >>> ds.labels.info.update(coords = {'type': 'pixel', 'LTRB'}) Copy to clipboard Note If the bounding box format is not specified, the visualizer will assume a YOLO format (fractional + CCWH) if the box coordinates are < 1 on average. Otherwise, it will assume the COCO format (pixel + LTWH). Appending bounding boxes¶ Bounding boxes can be appended as np.ndarrays or list or lists of arrays. Examples Appending one bounding box >>> box array([[462, 123, 238, 98]]) >>> ds.boxes.append(box) Copy to clipboard Appending sample with 3 bounding boxes >>> boxes array([[965, 110, 262, 77], [462, 123, 238, 98], [688, 108, 279, 116]]) >>> boxes.shape (3, 4) >>> ds.boxes.append(boxes) Copy to clipboard


Creating a bbox tensor¶

A bbox tensor can be created using Copy to clipboard Optional args: coords: A dictionary with keys 'type' and 'mode'. type: Specifies the units of bounding box coordinates. 'pixel': is in unit of pixels. 'fractional': is in units relative to the width and height of the image, such as in YOLO format. mode: Specifies the convention for the 4 coordinates 'LTRB': left_x, top_y, right_x, bottom_y 'LTWH': left_x, top_y, width, height 'CCWH': center_x, center_y, width, height dtype: Defaults to float32. sample_compression or chunk_compression. Supported compressions: Copy to clipboard You can also choose to set the class names after tensor creation. Copy to clipboard NoteIf the bounding box format is not specified, the visualizer will assume a YOLO format (fractional + CCWH) if the box coordinates are < 1 on average. Otherwise, it will assume the COCO format (pixel + LTWH).


Segmentation Mask Htype¶

Sample dimensions: (height, width) Segmentation masks are 2D representations of class labels where the numerical label data is encoded in an array of same shape as the image. The numerical values are indices of the list tensor.info.class_names. Creating a segment_mask tensor¶ A segment_mask tensor can be created using >>> classes = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle'] >>> ds.create_tensor('masks', htype='segment_mask', class_names=classes, sample_compression='lz4') Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression dtype: Defaults to uint32. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set the class names after tensor creation. >>> ds.labels.info.update(class_names = ['background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle']) Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4. Appending segmentation masks¶ Segmentation masks can be appended as np.ndarray. Examples >>> ds.masks.append(np.zeros((512, 512))) Copy to clipboard Note Since each pixel can only be labeled once, segmentation masks are not appropriate for datasets where objects might overlap, or where multiple objects within the same class must be distinguished. For these use cases, please use htype = 'binary_mask'.


Appending class labels¶

Class labels can be appended as int, str, np.ndarray or list of int or str. In case of strings, tensor.info.class_names is updated automatically. ExamplesAppending index Copy to clipboard Extending with list of indices Copy to clipboard Appending text labels Copy to clipboard


Appending segmentation masks¶

Segmentation masks can be appended as np.ndarray. Examples Copy to clipboard NoteSince each pixel can only be labeled once, segmentation masks are not appropriate for datasets where objects might overlap, or where multiple objects within the same class must be distinguished. For these use cases, please use htype = 'binary_mask'.


Creating a segment_mask tensor¶

A segment_mask tensor can be created using Copy to clipboard Optional args: class_names: This must be a list of strings. tensor.info.class_names will be set to this list. sample_compression or chunk_compression dtype: Defaults to uint32. Supported compressions: Copy to clipboard You can also choose to set the class names after tensor creation. Copy to clipboard NoteSince segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.


Binary Mask Htype¶

Sample dimensions: (height, width, # objects in a sample) Binary masks are similar to segmentation masks, except that each object is represented by a channel in the mask. Each channel in the mask encodes values for a single object. A pixel in a mask channel should have a value of 1 if the pixel of the image belongs to this object and 0 otherwise. The labels corresponding to the channels should be stored in an adjacent tensor of htype class_label, in which the number of labels at a given index is equal to the number of objects (number of channels) in the binary mask. Creating a binary_mask tensor¶ A binary_mask tensor can be created using >>> ds.create_tensor('masks', htype='binary_mask', sample_compression='lz4') Copy to clipboard Optional args: ref:sample_compression <sample_compression> or chunk_compression dtype: Defaults to bool. Supported compressions: >>> ['lz4'] Copy to clipboard Note Since segmentation masks often contain large amounts of data, it is recommended to compress them using lz4. Appending binary masks¶ Binary masks can be appended as np.ndarray. Examples Appending a binary mask with 5 objects >>> ds.masks.append(np.zeros((512, 512, 5), dtype='bool')) >>> ds.labels.append(['aeroplane', 'aeroplane', 'bottle', 'bottle', 'bird']) Copy to clipboard


Creating a binary_mask tensor¶

A binary_mask tensor can be created using Copy to clipboard Optional args: ref:sample_compression <sample_compression> or chunk_compression dtype: Defaults to bool. Supported compressions: Copy to clipboard NoteSince segmentation masks often contain large amounts of data, it is recommended to compress them using lz4.


COCO Keypoints Htype¶

Sample dimensions: (3 x # keypoints, # objects in a sample) COCO keypoints are a convention for storing points of interest in an image. Each keypoint consists of 3 values: x - coordinate, y - coordinate and v - visibility. A set of K keypoints of an object is represented as:[x1, y1, v1, x2, y2, v2, …, xk, yk, vk]The visibility v can be one of three values: 0: keypoint not in image. 1: keypoint in image but not visible. 2: keypoint in image and visible. Creating a keypoints_coco tensor¶ A keypoints_coco tensor can be created using >>> ds.create_tensor('keypoints', htype='keypoints_coco', keypoints=['knee', 'elbow', 'head'], connections=[[0, 1], [1, 2]]) Copy to clipboard Optional args: keypoints: List of strings describing the i th keypoint. tensor.info.keypoints will be set to this list. connections: List of strings describing which points should be connected by lines in the visualizer. sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: >>> ['lz4'] Copy to clipboard You can also choose to set keypoints and / or connections after tensor creation. >>> ds.keypoints.info.update(keypoints = ['knee', 'elbow',...]) >>> ds.keypoints.info.update(connections = [[0,1], [2,3], ...]) Copy to clipboard Appending keypoints¶ Keypoints can be appended as np.ndarray or list. Examples Appending keypoints sample with 3 keypoints and 4 objects >>> ds.keypoints.update(keypoints = ['left ear', 'right ear', 'nose']) >>> ds.keypoints.update(connections = [[0, 2], [1, 2]]) >>> kp_arr array([[465, 398, 684, 469], [178, 363, 177, 177], [ 2, 2, 2, 1], [454, 387, 646, 478], [177, 322, 137, 161], [ 2, 2, 2, 2], [407, 379, 536, 492], [271, 335, 150, 143], [ 2, 1, 2, 2]]) >>> kp_arr.shape (9, 4) >>> ds.keypoints.append(kp_arr) Copy to clipboard Warning In order to correctly use the keypoints and connections metadata, it is critical that all objects in every sample have the same number of K keypoints in the same order. For keypoints that are not present in an image, they can be stored with dummy coordinates of x = 0, y = 0, and v = 0, and the visibility will prevent them from being drawn in the visualizer.


Creating a keypoints_coco tensor¶

A keypoints_coco tensor can be created using Copy to clipboard Optional args: keypoints: List of strings describing the i th keypoint. tensor.info.keypoints will be set to this list. connections: List of strings describing which points should be connected by lines in the visualizer. sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: Copy to clipboard You can also choose to set keypoints and / or connections after tensor creation. Copy to clipboard


Appending keypoints¶

Keypoints can be appended as np.ndarray or list. ExamplesAppending keypoints sample with 3 keypoints and 4 objects Copy to clipboard WarningIn order to correctly use the keypoints and connections metadata, it is critical that all objects in every sample have the same number of K keypoints in the same order. For keypoints that are not present in an image, they can be stored with dummy coordinates of x = 0, y = 0, and v = 0, and the visibility will prevent them from being drawn in the visualizer.


Point Htype¶

Sample dimensions: (# points, 2) in case of 2-D (X, Y) co-ordinates or (# points, 3) in case of 3-D (X, Y, Z) co-ordinates of the point. Points does not contain a fixed mapping across samples between the point order and real-world objects (i.e., point 0 is an elbow, point 1 is a knee, etc.). If you require such a mapping, use COCO Keypoints Htype. Creating a point tensor¶ A point tensor can be created using >>> ds.create_tensor('points', htype='point', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to int32. Supported compressions: >>> ['lz4'] Copy to clipboard Appending point samples¶ Points can be appended as np.ndarray or list. Examples Appending 2 2-D points >>> ds.points.append([[0, 1], [1, 3]]) Copy to clipboard Appending 2 3-D points >>> ds.points.append(np.zeros((2, 3))) Copy to clipboard


Appending binary masks¶

Binary masks can be appended as np.ndarray. ExamplesAppending a binary mask with 5 objects Copy to clipboard


Appending bounding boxes¶

Bounding boxes can be appended as np.ndarrays or list or lists of arrays. ExamplesAppending one bounding box Copy to clipboard Appending sample with 3 bounding boxes Copy to clipboard


Appending point samples¶

Points can be appended as np.ndarray or list. ExamplesAppending 2 2-D points Copy to clipboard Appending 2 3-D points Copy to clipboard


Polygon Htype¶

Sample dimensions: (# polygons, # points per polygon, # co-ordinates per point) Each sample in a tensor of polygon htype is a list of polygons. Each polygon is a list / array of points. All points in a sample should have the same number of co-ordinates (eg., cannot mix 2-D points with 3-D points). Different samples can have different number of polygons. Different polygons can have different number of points. Creating a polygon tensor¶ A polygon tensor can be created using >>> ds.create_tensor('polygons', htype='polygon', sample_compression=None) Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to float32. Supported compressions: >>> ['lz4'] Copy to clipboard Appending polygons¶ Polygons can be appended as a list of list of tuples or np.ndarray. Examples Appending polygons with 2-D points >>> poly1 = [(1, 2), (2, 3), (3, 4)] >>> poly2 = [(10, 12), (14, 19)] >>> poly3 = [(33, 32), (54, 67), (67, 43), (56, 98)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with 3-D points >>> poly1 = [(10, 2, 9), (12, 3, 8), (12, 10, 4)] >>> poly2 = [(10, 1, 8), (5, 17, 11)] >>> poly3 = [(33, 33, 31), (45, 76, 13), (60, 24, 17), (67, 87, 83)] >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard Appending polygons with numpy arrays >>> import numpy as np >>> sample = np.random.randint(0, 10, (5, 7, 2)) # 5 polygons with 7 points >>> ds.polygons.append(sample) Copy to clipboard >>> import numpy as np >>> poly1 = np.random.randint(0, 10, (5, 2)) >>> poly2 = np.random.randint(0, 10, (8, 2)) >>> poly3 = np.random.randint(0, 10, (3, 2)) >>> sample = [poly1, poly2, poly3] >>> ds.polygons.append(sample) Copy to clipboard


Creating a polygon tensor¶

A polygon tensor can be created using Copy to clipboard Optional args: sample_compression or chunk_compression dtype: Defaults to float32. Supported compressions: Copy to clipboard


Appending polygons¶

Polygons can be appended as a list of list of tuples or np.ndarray. ExamplesAppending polygons with 2-D points Copy to clipboard Appending polygons with 3-D points Copy to clipboard Appending polygons with numpy arrays Copy to clipboard Copy to clipboard