Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums
By A Mystery Man Writer
Description
Excuse me @Bin_Zhao_NV @Morganh I’ve changed gpus from Tesla P100 to Tesla V100 and tried to train Tao Toolkit UNet model with 4 gpus in version v4.0.0 and v4.0.1 again. However. I still got the error message: device CUDA:0 not supported by XLA service while setting up XLA_GPU_JIT device number 0. This is the result in the process of training UNet when I ran the command nvidia-smi. Is this a bug for Tao Toolkit v4.0.0 and v4.0.1 ? When I trained UNet in the version v3.22.05, it seemed tha
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://developer-blogs.nvidia.com/wp-content/uploads/2021/08/tao-workflow-1-625x351.png)
Building and Deploying Conversational AI Models Using NVIDIA TAO Toolkit
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://global.discourse-cdn.com/nvidia/optimized/3X/a/f/afe0e80240c45ab7cbc67eb0d8cacdbe968df666_2_689x177.png)
Resnet50 is not UTF-8 encoded - TAO Toolkit - NVIDIA Developer Forums
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://global.discourse-cdn.com/nvidia/original/3X/1/0/1016a336f7d5c18c07df5c5306797cac64c30974.png)
The training process of Tao-Toolkit-API unet is always in Inf status - TAO Toolkit - NVIDIA Developer Forums
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://global.discourse-cdn.com/nvidia/original/3X/f/f/ff4f35ed7e91f1a488eb686f793c35d6ff570d09.png)
The training process of Tao-Toolkit-API unet is always in Inf status - TAO Toolkit - NVIDIA Developer Forums
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://community.arm.com/cfs-file/__key/communityserver-blogs-components-weblogfiles/00-00-00-38-23/1031.Smart-pruning-post-image-_2800_1_2900_.jpg)
Smart Pruning: Improve mobile ML performance - AI and ML blog - Arm Community blogs - Arm Community
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://www.biorxiv.org/content/biorxiv/early/2023/04/30/2023.04.30.538453/F3.large.jpg)
Segmentation of 3D blood vessel networks using unsupervised deep learning
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://developer-blogs.nvidia.com/wp-content/uploads/2023/01/WB-experiment-tracking-dashboard-screenshot.png)
Accelerating AI Development with NVIDIA TAO Toolkit and Weights & Biases
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://global.discourse-cdn.com/nvidia/original/4X/e/8/f/e8f7aa7708574c0834b3f8b8b628f695be85488d.png)
Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://global.discourse-cdn.com/nvidia/original/3X/4/5/4548a8b2cd3b194414608e43b4aae96b22c51de7.jpeg)
Problems encountered in training unet and inference unet - #27 by Morganh - TAO Toolkit - NVIDIA Developer Forums
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://docscontent.nvidia.com/dims4/default/4643455/2147483647/strip/true/crop/11811x6834+0+0/resize/1440x833!/quality/90/?url=https%3A%2F%2Fk3-prod-nvidia-docs.s3.us-west-2.amazonaws.com%2Fbrightspot%2Fsphinx%2F0000018e-5279-dda8-a78f-d37fe40e0000%2Ftao%2Ftao-toolkit%2F_images%2Ftao_workflow_detectnet_v2.png)
DetectNet_v2 - NVIDIA Docs
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://d29g4g2dyqv443.cloudfront.net/sites/default/files/akamai/TAO/nvidia-ai-enterprise-630x354.jpg)
TAO Toolkit NVIDIA Developer
![Cannot train Tao Toolkit UNet model in version v4.0.0 and v4.0.1 - TAO Toolkit - NVIDIA Developer Forums](https://docscontent.nvidia.com/dims4/default/dae0d08/2147483647/strip/true/crop/7457x5321+0+0/resize/1440x1028!/quality/90/?url=https%3A%2F%2Fk3-prod-nvidia-docs.s3.us-west-2.amazonaws.com%2Fbrightspot%2Fsphinx%2F0000018c-4c1e-dd8a-afaf-ce9ed4c10000%2Ftao%2Ftao-toolkit%2F_images%2Ftao_overview.png)
Integrating TAO Models into DeepStream - NVIDIA Docs
from
per adult (price varies by group size)