{"id":1292,"date":"2024-10-17T12:22:21","date_gmt":"2024-10-17T12:22:21","guid":{"rendered":"https:\/\/zahiralam.com\/blog\/?p=1292"},"modified":"2024-10-17T13:45:18","modified_gmt":"2024-10-17T13:45:18","slug":"bring-images-to-life-install-hallo2-on-ubuntu-for-4k-long-duration-talking-videos","status":"publish","type":"post","link":"https:\/\/zahiralam.com\/blog\/bring-images-to-life-install-hallo2-on-ubuntu-for-4k-long-duration-talking-videos\/","title":{"rendered":"Bring Images to Life: Install Hallo2 on Ubuntu for 4K, Long-Duration Talking Videos"},"content":{"rendered":"\n<p>\n\n\n\n<p><strong>Hallo2<\/strong> is a powerful framework designed for creating long-duration, high-resolution, audio-driven portrait image animations. It stands out for its ability to generate <strong>4K videos<\/strong> for animations lasting <strong>up to or beyond 1 hour<\/strong>, making it a go-to tool for extended animations with stunning detail. This tutorial will guide you step-by-step through the process of installing and running Hallo2 on <strong>Ubuntu<\/strong>, so you can leverage its full potential for your AI animation projects.\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Prerequisites<\/strong><\/h3>\n\n\n\n<p>Before starting, ensure that you have:\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Operating System<\/strong>: Ubuntu 20.04 or Ubuntu 22.04<\/li>\n\n\n\n<li><strong>CUDA<\/strong>: CUDA 11.8 (for GPU acceleration)<\/li>\n\n\n\n<li><strong>GPU<\/strong>: Tested with NVIDIA A100 (other CUDA-enabled GPUs may also work)<\/li>\n\n\n\n<li><strong>Python<\/strong>: Python 3.10 or higher<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>1. Clone the Hallo2 Repository<\/strong><\/h3>\n\n\n\n<p>First, clone the Hallo2 repository from GitHub to obtain the necessary code, scripts, and configuration files.\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-1\"># Clone the Hallo2 repository\ngit clone https:\/\/github.com\/fudan-generative-vision\/hallo2.git\n\n# Navigate into the project directory\ncd hallo2<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#%23%20Clone%20the%20Hallo2%20repository%0Agit%20clone%20https%3A%2F%2Fgithub.com%2Ffudan-generative-vision%2Fhallo2.git%0A%0A%23%20Navigate%20into%20the%20project%20directory%0Acd%20hallo2\">\n                            <button class=\"copy-button\" data-label=\"# Clone the Hallo2 repository\ngit clone https:\/\/github.com\/fudan-generative-vision\/hallo2.git\n\n# Navigate into the project directory\ncd hallo2\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>This step is essential as it provides all the scripts you will need to run Hallo2.\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>2. Install Git LFS<\/strong><\/h3>\n\n\n\n<p>Git LFS (Large File Storage) is required to download large model files. Install it using the following commands:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-2\"># Install Git LFS\nsudo apt-get install git-lfs\n\n# Set up Git LFS\ngit lfs install<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#%23%20Install%20Git%20LFS%0Asudo%20apt-get%20install%20git-lfs%0A%0A%23%20Set%20up%20Git%20LFS%0Agit%20lfs%20install\">\n                            <button class=\"copy-button\" data-label=\"# Install Git LFS\nsudo apt-get install git-lfs\n\n# Set up Git LFS\ngit lfs install\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>3. Install Conda<\/strong><\/h3>\n\n\n\n<p>If you don&#8217;t have Conda installed, you can install Miniconda to create and manage the Python environment. Follow these steps:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-3\"># Download Miniconda installer\nwget https:\/\/repo.anaconda.com\/miniconda\/Miniconda3-latest-Linux-x86_64.sh\n\n# Run the installer\nbash Miniconda3-latest-Linux-x86_64.sh\n\n# Restart your terminal<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#%23%20Download%20Miniconda%20installer%0Awget%20https%3A%2F%2Frepo.anaconda.com%2Fminiconda%2FMiniconda3-latest-Linux-x86_64.sh%0A%0A%23%20Run%20the%20installer%0Abash%20Miniconda3-latest-Linux-x86_64.sh%0A%0A%23%20Restart%20your%20terminal\">\n                            <button class=\"copy-button\" data-label=\"# Download Miniconda installer\nwget https:\/\/repo.anaconda.com\/miniconda\/Miniconda3-latest-Linux-x86_64.sh\n\n# Run the installer\nbash Miniconda3-latest-Linux-x86_64.sh\n\n# Restart your terminal\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>After installing Conda, create a Conda environment for Hallo2 with Python 3.10:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-4\">conda create -n hallo python=3.10\nconda activate hallo<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#conda%20create%20-n%20hallo%20python%3D3.10%0Aconda%20activate%20hallo\">\n                            <button class=\"copy-button\" data-label=\"conda create -n hallo python=3.10\nconda activate hallo\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>4. Install Required Python Packages<\/strong><\/h3>\n\n\n\n<p>With your Conda environment active, install the required Python packages. Start by installing PyTorch with CUDA 11.8 support:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-5\">pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 --index-url https:\/\/download.pytorch.org\/whl\/cu118<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#pip%20install%20torch%3D%3D2.2.2%20torchvision%3D%3D0.17.2%20torchaudio%3D%3D2.2.2%20--index-url%20https%3A%2F%2Fdownload.pytorch.org%2Fwhl%2Fcu118\">\n                            <button class=\"copy-button\" data-label=\"pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 --index-url https:\/\/download.pytorch.org\/whl\/cu118\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>Next, install the remaining dependencies by running:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-6\">pip install -r requirements.txt<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#pip%20install%20-r%20requirements.txt\">\n                            <button class=\"copy-button\" data-label=\"pip install -r requirements.txt\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>Also, install <strong>FFmpeg<\/strong>, which is necessary for video processing:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-7\">sudo apt-get install ffmpeg<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#sudo%20apt-get%20install%20ffmpeg\">\n                            <button class=\"copy-button\" data-label=\"sudo apt-get install ffmpeg\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>5. Download Pretrained Models<\/strong><\/h3>\n\n\n\n<p>Hallo2 requires pretrained models, which you can download from HuggingFace. Run the following commands to clone the pretrained models:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-8\">git clone https:\/\/huggingface.co\/fudan-generative-ai\/hallo2 pretrained_models<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#git%20clone%20https%3A%2F%2Fhuggingface.co%2Ffudan-generative-ai%2Fhallo2%20pretrained_models\">\n                            <button class=\"copy-button\" data-label=\"git clone https:\/\/huggingface.co\/fudan-generative-ai\/hallo2 pretrained_models\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>Alternatively, you can download each pretrained model from their respective repositories. Make sure to organize them in the following directory structure:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-9\">.\/pretrained_models\/\n|-- audio_separator\/\n|-- CodeFormer\/\n|-- face_analysis\/\n|-- facelib\n|-- hallo2\n|-- motion_module\/\n|-- realesrgan\n|-- sd-vae-ft-mse\/\n|-- stable-diffusion-v1-5\/\n|-- wav2vec\/<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#.%2Fpretrained_models%2F%0A%7C--%20audio_separator%2F%0A%7C--%20CodeFormer%2F%0A%7C--%20face_analysis%2F%0A%7C--%20facelib%0A%7C--%20hallo2%0A%7C--%20motion_module%2F%0A%7C--%20realesrgan%0A%7C--%20sd-vae-ft-mse%2F%0A%7C--%20stable-diffusion-v1-5%2F%0A%7C--%20wav2vec%2F\">\n                            <button class=\"copy-button\" data-label=\".\/pretrained_models\/\n|-- audio_separator\/\n|-- CodeFormer\/\n|-- face_analysis\/\n|-- facelib\n|-- hallo2\n|-- motion_module\/\n|-- realesrgan\n|-- sd-vae-ft-mse\/\n|-- stable-diffusion-v1-5\/\n|-- wav2vec\/\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>6. Edit the Configuration File<\/strong><\/h3>\n\n\n\n<p>The next step is to configure the paths in the YAML file. The default configuration file (<code>configs\/inference\/long.yaml<\/code>) provides example paths for the source image and driving audio:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-10\">source_image: .\/examples\/reference_images\/1.jpg\ndriving_audio: .\/examples\/driving_audios\/1.wav<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#source_image%3A%20.%2Fexamples%2Freference_images%2F1.jpg%0Adriving_audio%3A%20.%2Fexamples%2Fdriving_audios%2F1.wav\">\n                            <button class=\"copy-button\" data-label=\"source_image: .\/examples\/reference_images\/1.jpg\ndriving_audio: .\/examples\/driving_audios\/1.wav\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>You can replace these with your own files by updating the <code>source_image<\/code> and <code>driving_audio<\/code> fields with the appropriate paths.\n\n\n\n<p>Make sure the paths for pretrained models are correct in the configuration, and check that the <code>save_path<\/code> is where you want to store the results. The default is:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-11\">save_path: .\/output_long\/debug\/<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#save_path%3A%20.%2Foutput_long%2Fdebug%2F\">\n                            <button class=\"copy-button\" data-label=\"save_path: .\/output_long\/debug\/\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>7. Running Inference<\/strong><\/h3>\n\n\n\n<p>Now you are ready to run inference and generate animations based on your source image and driving audio.\n\n\n\n<p>Run the following command to start the animation generation process:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-12\">python scripts\/inference_long.py --config .\/configs\/inference\/long.yaml<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#python%20scripts%2Finference_long.py%20--config%20.%2Fconfigs%2Finference%2Flong.yaml\">\n                            <button class=\"copy-button\" data-label=\"python scripts\/inference_long.py --config .\/configs\/inference\/long.yaml\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>This script will generate an animation based on the image and audio, and save the result to the location specified in the <code>save_path<\/code>.\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>8. (Optional) High-Resolution Video Upsampling<\/strong><\/h3>\n\n\n\n<p>To further enhance the quality of the video, you can run the high-resolution upsampling script. This improves both the background and face areas in the video:\n\n\n\n<div class=\"code-block-container\">\n                        <pre class=\"wp-block-code\"><code id=\"code-13\">python scripts\/video_sr.py --input_path [input_video] --output_path [output_dir] --bg_upsampler realesrgan --face_upsample -w 1 -s 4<\/code><\/pre>\n                        <amp-iframe sandbox=\"allow-scripts\" width=\"94\" height=\"72\" frameborder=\"0\" \n                                    src=\"https:\/\/zahiralam.com\/blog\/wp-content\/plugins\/amp-copy-code-button\/copier.html#python%20scripts%2Fvideo_sr.py%20--input_path%20%5Binput_video%5D%20--output_path%20%5Boutput_dir%5D%20--bg_upsampler%20realesrgan%20--face_upsample%20-w%201%20-s%204\">\n                            <button class=\"copy-button\" data-label=\"python scripts\/video_sr.py --input_path [input_video] --output_path [output_dir] --bg_upsampler realesrgan --face_upsample -w 1 -s 4\"  placeholder disabled>Copy<\/button>\n                        <\/amp-iframe>\n                    <\/div>\n\n\n\n<p>Replace <code>[input_video]<\/code> with the path to the generated video and specify the output directory in <code>[output_dir]<\/code>. This command will enhance the video and save the result in the specified folder.\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>9. Troubleshooting<\/strong><\/h3>\n\n\n\n<p>Here are some common issues and their solutions:\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>CUDA not found<\/strong>: Make sure CUDA 11.8 is installed correctly by running <code>nvcc --version<\/code>. If not, follow a CUDA installation guide.<\/li>\n\n\n\n<li><strong>Pretrained model not found<\/strong>: Verify that all pretrained models are placed in the correct folders under <code>pretrained_models<\/code>.<\/li>\n\n\n\n<li><strong>Slow inference<\/strong>: Ensure that you are using a CUDA-enabled GPU for faster inference. Running on a CPU will significantly slow down the process.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>10. Hardware Considerations<\/strong><\/h3>\n\n\n\n<p>To get the best performance out of Hallo2, ensure that you are using:\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>GPU<\/strong>: A CUDA-compatible GPU is recommended for faster inference. The repository has been tested with NVIDIA A100, but other GPUs should work if they support CUDA.<\/li>\n\n\n\n<li><strong>Memory<\/strong>: Depending on the model size and image resolution, your GPU should have at least 16GB of memory to run the models smoothly.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>11. Viewing Results<\/strong><\/h3>\n\n\n\n<p>Once inference is complete, the generated animation will be saved in the directory you specified in the configuration file. You can open and view the resulting video using a media player like VLC.\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Conclusion<\/strong><\/h3>\n\n\n\n<p>By following this tutorial, you should now be able to install, configure, and run Hallo2 on your Ubuntu system. With the pretrained models and scripts in place, you can generate high-quality animations using your own images and audio. For further customization, feel free to explore the configuration file and experiment with different settings.\n","protected":false},"excerpt":{"rendered":"<p>Hallo2 is a powerful framework designed for creating long-duration, high-resolution, audio-driven portrait image animations. It stands out for its ability to generate 4K videos for [&#8230;]<\/p>\n","protected":false},"author":1,"featured_media":1300,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24],"tags":[245,246,244,18],"class_list":["post-1292","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","tag-ai-portrait-animation","tag-face-animation","tag-hallo2","tag-ubuntu"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/posts\/1292","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/comments?post=1292"}],"version-history":[{"count":8,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/posts\/1292\/revisions"}],"predecessor-version":[{"id":1303,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/posts\/1292\/revisions\/1303"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/media\/1300"}],"wp:attachment":[{"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/media?parent=1292"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/categories?post=1292"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zahiralam.com\/blog\/wp-json\/wp\/v2\/tags?post=1292"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}