Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release, but uses synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. The original Depth Anything model was introduced in the paper Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data by Lihe Yang et al., and was first released in this repository. Depth Anything V2 leverages the DPT architecture with a DINOv2 backbone. The model is trained on ~600K synthetic labeled images and ~62 million real unlabeled images, obtaining state-of-the-art results for both relative and absolute depth estimation.
翻訳待ち・原文を表示
入力 画像→ 出力 画像
We present a foundation model for zero-shot metric monocular depth estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with unparalleled sharpness and high-frequency details. The predictions are metric, with absolute scale, without relying on the availability of metadata such as camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map in 0.3 seconds on a standard GPU. These characteristics are enabled by a number of technical contributions, including an efficient multi-scale vision transformer for dense prediction, a training protocol that combines real and synthetic datasets to achieve high metric accuracy alongside fine boundary tracing, dedicated evaluation metrics for boundary accuracy in estimated depth maps, and state-of-the-art focal length estimation from a single image. Depth Pro was introduced in **Depth Pro: Sharp Monocular Metric Depth in Less Than a Second**, by *Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R. Richter, and Vladlen Koltun*. The checkpoint in this repository is a reference implementation, which has been re-trained. Its performance is close to the model reported in the paper but does not match it exactly.
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This is the transformers version of DepthPro, a foundation model for zero-shot metric monocular depth estimation, designed to generate high-resolution depth maps with remarkable sharpness and fine-grained details. For the checkpoint compatible with the original codebase, please check this repo. - DepthPro: Monocular Depth Estimation - Table of Contents - Model Details - Model Sources - How to Get Started with the Model - Training Details - Training Data - Preprocessing - Training Hyperparameters - Evaluation - Model Architecture and Objective - Citation - Model Card Authors DepthPro is a foundation model for zero-shot metric monocular depth estimation, designed to generate high-resolution depth maps with remarkable sharpness and fine-grained details. It employs a multi-scale Vision Transformer (ViT)-based architecture, where images are downsampled, divided into patches, and processed using a shared Dinov2 encoder. The extracted patch-level features are merged, upsampled, and refined using a DPT-like fusion stage, enabling precise depth estimation.
翻訳待ち・原文を表示
入力 画像→ 出力 画像
- https://github.com/ByteDance-Seed/Depth-Anything-3 - https://huggingface.co/depth-anything/DA3-BASE - https://huggingface.co/depth-anything/DA3METRIC-LARGE - https://huggingface.co/depth-anything/DA3MONO-LARGE - https://huggingface.co/depth-anything/DA3-SMALL Depth Map: Depth Anything 3 Before After Depth Estimation: Depth Anything 3 Before After
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Giant model for multi-view depth estimation, camera pose estimation, and 3D Gaussian estimation. This is the flagship foundation model with unified depth-ray representation. - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Giant model for multi-view depth estimation, camera pose estimation, and 3D Gaussian estimation. This is the flagship foundation model with unified depth-ray representation. 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501 🏆 Depth Anything 3 significantly outperforms: - **Depth Anything 2** for monocular depth estimation - **VGGT** for multi-view depth estimation and pose estimation
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Large model for multi-view depth estimation and camera pose estimation. Foundation model with unified depth-ray representation. - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Large model for multi-view depth estimation and camera pose estimation. Foundation model with unified depth-ray representation. - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Small model for multi-view depth estimation and camera pose estimation. Efficient foundation model with unified depth-ray representation. - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Metric Large model specialized for metric depth estimation in monocular settings, ideal for applications requiring real-world scale. Canonical metric depth; multiplying by focal length gives metric depth. - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Monocular Large model for high-quality relative monocular depth estimation. Unlike disparity-based models (e.g., Depth Anything 2), it directly predicts depth, resulting in superior geometric accuracy. - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only 💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Nested model combining the any-view Giant model with the metric Large model for metric-scale visual geometry reconstruction. This is our recommended model that combines all capabilities. - ✅ Relative Depth - ✅ Pose Estimation - ✅ Pose Conditioning - ✅ 3D Gaussians - ✅ Metric Depth - ✅ Sky Segmentation - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only
翻訳待ち・原文を表示
入力 画像→ 出力 画像
DA3 Nested model combining the any-view Giant model with the metric Large model for metric-scale visual geometry reconstruction. This is our recommended model that combines all capabilities. - ✅ Relative Depth - ✅ Pose Estimation - ✅ Pose Conditioning - ✅ 3D Gaussians - ✅ Metric Depth - ✅ Sky Segmentation - **Developed by:** ByteDance Seed Team - **Model Type:** Vision Transformer for Visual Geometry - **Architecture:** Plain transformer with unified depth-ray representation - **Training Data:** Public academic datasets only
翻訳待ち・原文を表示
入力 画像→ 出力 画像
Depth Anything V2 is trained from 595K synthetic labeled images and 62M+ real unlabeled images, providing the most capable monocular depth estimation (MDE) model with the following features: - more fine-grained details than Depth Anything V1 - more robust than Depth Anything V1 and SD-based models (e.g., Marigold, Geowizard) - more efficient (10x faster) and more lightweight than SD-based models - impressive fine-tuned performance with our pre-trained models ```bibtex @article{depth_anything_v2, title={Depth Anything V2}, author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang}, journal={arXiv:2406.09414}, year={2024} } @inproceedings{depth_anything_v1, title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data}, author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang}, booktitle={CVPR}, year={2024} }
翻訳待ち・原文を表示
入力 画像→ 出力 画像
Depth Anything V2 is trained from 595K synthetic labeled images and 62M+ real unlabeled images, providing the most capable monocular depth estimation (MDE) model with the following features: - more fine-grained details than Depth Anything V1 - more robust than Depth Anything V1 and SD-based models (e.g., Marigold, Geowizard) - more efficient (10x faster) and more lightweight than SD-based models - impressive fine-tuned performance with our pre-trained models Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release, but uses synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. The original Depth Anything model was introduced in the paper Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data by Lihe Yang et al., and was first released in this repository. Depth Anything V2 leverages the DPT architecture with a DINOv2 backbone.
翻訳待ち・原文を表示
入力 画像→ 出力 画像
Depth Anything V2 is trained from 595K synthetic labeled images and 62M+ real unlabeled images, providing the most capable monocular depth estimation (MDE) model with the following features: - more fine-grained details than Depth Anything V1 - more robust than Depth Anything V1 and SD-based models (e.g., Marigold, Geowizard) - more efficient (10x faster) and more lightweight than SD-based models - impressive fine-tuned performance with our pre-trained models Download the model first and put it under the `checkpoints` directory. ```bibtex @article{depth_anything_v2, title={Depth Anything V2}, author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang}, journal={arXiv:2406.09414}, year={2024} }
翻訳待ち・原文を表示
入力 画像→ 出力 画像
Depth Anything V2 is trained from 595K synthetic labeled images and 62M+ real unlabeled images, providing the most capable monocular depth estimation (MDE) model with the following features: - more fine-grained details than Depth Anything V1 - more robust than Depth Anything V1 and SD-based models (e.g., Marigold, Geowizard) - more efficient (10x faster) and more lightweight than SD-based models - impressive fine-tuned performance with our pre-trained models Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release, but uses synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. The original Depth Anything model was introduced in the paper Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data by Lihe Yang et al., and was first released in this repository. Depth Anything V2 leverages the DPT architecture with a DINOv2 backbone.
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This model represents a fine-tuned version of Depth Anything V2 for indoor metric depth estimation using the synthetic Hypersim datasets. Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release but employs synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. This fine-tuned version for metric depth estimation was first released in this repository. **Six metric depth models** of three scales for indoor and outdoor scenes, respectively, were released and are available:
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This model represents a fine-tuned version of Depth Anything V2 for indoor metric depth estimation using the synthetic Hypersim datasets. Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release but employs synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. This fine-tuned version for metric depth estimation was first released in this repository. **Six metric depth models** of three scales for indoor and outdoor scenes, respectively, were released and are available:
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This model represents a fine-tuned version of Depth Anything V2 for indoor metric depth estimation using the synthetic Hypersim datasets. Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release but employs synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. This fine-tuned version for metric depth estimation was first released in this repository. **Six metric depth models** of three scales for indoor and outdoor scenes, respectively, were released and are available:
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This model represents a fine-tuned version of Depth Anything V2 for outdoor metric depth estimation using the synthetic Virtual KITTI datasets. Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release but employs synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. This fine-tuned version for metric depth estimation was first released in this repository. **Six metric depth models** of three scales for indoor and outdoor scenes, respectively, were released and are available:
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This model represents a fine-tuned version of Depth Anything V2 for outdoor metric depth estimation using the synthetic Virtual KITTI datasets. Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release but employs synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. This fine-tuned version for metric depth estimation was first released in this repository. **Six metric depth models** of three scales for indoor and outdoor scenes, respectively, were released and are available:
翻訳待ち・原文を表示
入力 画像→ 出力 画像
This model represents a fine-tuned version of Depth Anything V2 for outdoor metric depth estimation using the synthetic Virtual KITTI datasets. Depth Anything V2 was introduced in the paper of the same name by Lihe Yang et al. It uses the same architecture as the original Depth Anything release but employs synthetic data and a larger capacity teacher model to achieve much finer and robust depth predictions. This fine-tuned version for metric depth estimation was first released in this repository. **Six metric depth models** of three scales for indoor and outdoor scenes, respectively, were released and are available:
翻訳待ち・原文を表示
入力 画像→ 出力 画像
Depth Anything V2 is trained from 595K synthetic labeled images and 62M+ real unlabeled images, providing the most capable monocular depth estimation (MDE) model with the following features: - more fine-grained details than Depth Anything V1 - more robust than Depth Anything V1 and SD-based models (e.g., Marigold, Geowizard) - more efficient (10x faster) and more lightweight than SD-based models - impressive fine-tuned performance with our pre-trained models ```bibtex @article{depth_anything_v2, title={Depth Anything V2}, author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang}, journal={arXiv:2406.09414}, year={2024} } @inproceedings{depth_anything_v1, title={Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data}, author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang}, booktitle={CVPR}, year={2024} }
翻訳待ち・原文を表示
入力 画像→ 出力 画像
