{"id":277,"date":"2025-04-02T17:01:22","date_gmt":"2025-04-02T09:01:22","guid":{"rendered":"https:\/\/home.miawy.com\/?p=277"},"modified":"2025-04-24T15:37:21","modified_gmt":"2025-04-24T07:37:21","slug":"self-deploy-dify-xinference-ollama","status":"publish","type":"post","link":"https:\/\/home.miawy.com\/?p=277","title":{"rendered":"Self-deploy Dify + Xinference \/ Ollama"},"content":{"rendered":"\n<div class=\"wp-block-aioseo-table-of-contents\"><ul><li><a href=\"#aioseo-check-environment\">Check Environment<\/a><ul><li><a href=\"#aioseo-check-docker-and-docker\">Check Docker and Docker<\/a><\/li><li><a href=\"#aioseo-optional-change-root-directory-of-docker-data\">(Optional) Change root directory of Docker data<\/a><\/li><li><a href=\"#aioseo-nvidia-container-toolkit\">NVIDIA Container Toolkit<\/a><\/li><\/ul><\/li><li><a href=\"#aioseo-install-and-config-dify\">Install and config Dify<\/a><\/li><li><a href=\"#aioseo-install-and-config-xinference\">[OPTION 1] Install and config Xinference<\/a><ul><li><a href=\"#aioseo-install-xinference\">Install Xinference<\/a><\/li><li><a href=\"#aioseo-start-xinference\">Start Xinference<\/a><\/li><li><a href=\"#aioseo-register-model\">Register model<\/a><\/li><li><a href=\"#aioseo-launch-model\">Launch model<\/a><\/li><\/ul><\/li><li><a href=\"#aioseo-option-2-install-and-config-ollama\">[OPTION 2] Install and config Ollama<\/a><\/li><li><a href=\"#aioseo-integrate-xinference-in-dify\">Integrate Xinference\/Ollama in Dify<\/a><\/li><li><a href=\"#aioseo-integrate-ollama-in-dify\">FAQ<\/a><ul><li><a href=\"#aioseo-already-installed-docker-and-docker-compose-but-systemctl-status-docker-got-error\">Already installed Docker and Docker Compose, but systemctl status docker got error.<\/a><\/li><li><a href=\"#aioseo-could-not-register\">Could not find model when register model.<\/a><\/li><\/ul><\/li><\/ul><\/div>\n\n\n\n<p>This tutorial will show how to build your own RAG: deploy Dify + Xinference\/Ollama in local server.<\/p>\n\n\n\n<p><a href=\"https:\/\/docs.dify.ai\" target=\"_blank\" rel=\"noreferrer noopener\">What is Dify?<\/a><\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-check-environment\">Check Environment<\/h1>\n\n\n\n<ul class=\"wp-block-list\">\n<li>NVIDIA Driver version &gt;= 550<\/li>\n\n\n\n<li>CUDA version &gt;= 12.4<\/li>\n\n\n\n<li>Docker version &gt;= 19.03 (recommend &gt; 25)<\/li>\n\n\n\n<li>Docker Compose version &gt;= 1.28 (recommend &gt; v2)<\/li>\n\n\n\n<li>NVIDIA Container Toolkit<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading has-medium-font-size\" id=\"aioseo-check-docker-and-docker\">Check Docker and Docker<\/h2>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# Check Docker and Docker Compose version\ndocker --version\ndocker compose version\n\n# Check if docker is running normally\nsystemctl status docker\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># Check Docker and Docker Compose version<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #569CD6\">--version<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">compose<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">version<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># Check if docker is running normally<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">systemctl<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">status<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">docker<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"865\" height=\"268\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image.png\" alt=\"\" class=\"wp-image-311\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image.png 865w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-300x93.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-768x238.png 768w\" sizes=\"auto, (max-width: 865px) 100vw, 865px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"aioseo-optional-change-root-directory-of-docker-data\">(Optional) Change root directory of Docker data<\/h2>\n\n\n\n<p>Make sure there is enough space for Docker as the you will pull images from Docker and the images size is usually large.<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# Check current Docker root directory\ndocker info | grep &quot;Docker Root Dir&quot;\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># Check current Docker root directory<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">info<\/span><span style=\"color: #D4D4D4\"> | <\/span><span style=\"color: #DCDCAA\">grep<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">&quot;Docker Root Dir&quot;<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<p>Edit \/etc\/docker\/daemon.json, (if not exists, then create one). Modify the data-root path then save:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# vi \/etc\/docker\/daemon.json\n{\n  &quot;data-root&quot;: &quot;\/new\/path\/to\/docker\/data&quot;\n}\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># vi \/etc\/docker\/daemon.json<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">{<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">&quot;data-root&quot;<\/span><span style=\"color: #DCDCAA\">:<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">&quot;\/new\/path\/to\/docker\/data&quot;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">}<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">ShellScript<\/span><\/div>\n\n\n\n<p>Save and restart Docker service to reload the config:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"sudo systemctl restart docker\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #DCDCAA\">sudo<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">systemctl<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">restart<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">docker<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<p>Verify if it is effective, the output should be the path you set:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"docker info | grep &quot;Docker Root Dir&quot;\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">info<\/span><span style=\"color: #D4D4D4\"> | <\/span><span style=\"color: #DCDCAA\">grep<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">&quot;Docker Root Dir&quot;<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"aioseo-nvidia-container-toolkit\">NVIDIA Container Toolkit<\/h2>\n\n\n\n<p>Enable Docker container to use GPU. If skip this step, you might not able to access GPU resources inside your container.<\/p>\n\n\n\n<p>NVIDIA Installation guide could be found <a href=\"https:\/\/docs.nvidia.com\/datacenter\/cloud-native\/container-toolkit\/latest\/install-guide.html#installing-with-apt\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"869\" height=\"396\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-2.png\" alt=\"\" class=\"wp-image-344\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-2.png 869w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-2-300x137.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-2-768x350.png 768w\" sizes=\"auto, (max-width: 869px) 100vw, 869px\" \/><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-install-and-config-dify\">Install and config Dify<\/h1>\n\n\n\n<p>Dify installation guide could be found <a href=\"https:\/\/docs.dify.ai\/zh-hans\/getting-started\/install-self-hosted\/docker-compose\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>. Below I use source code from Github and Docker compose to start the services as it is the easiest way to deploy, and even upgrade in the future.<\/p>\n\n\n\n<p>Download source code from <a href=\"https:\/\/github.com\/langgenius\/dify\" target=\"_blank\" rel=\"noreferrer noopener\">Github<\/a>, or clone the source code:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"git clone https:\/\/github.com\/langgenius\/dify.git\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #DCDCAA\">git<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">clone<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">https:\/\/github.com\/langgenius\/dify.git<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<p>Go to dify\/docker, and modify the config if it needs. The environment variables explanation could be found <a href=\"https:\/\/docs.dify.ai\/getting-started\/install-self-hosted\/environments\">here<\/a>.<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# Copy the environment config\ncp .env.example .env\n\n# Modify the config file\nvi .env\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># Copy the environment config<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">cp<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">.env.example<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">.env<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># Modify the config file<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">vi<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">.env<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<p>Usually I will change the service port instead of using the default port, and you will use this port to log in later:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"722\" height=\"133\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-3.png\" alt=\"\" class=\"wp-image-369\" style=\"width:391px;height:auto\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-3.png 722w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-3-300x55.png 300w\" sizes=\"auto, (max-width: 722px) 100vw, 722px\" \/><\/figure>\n\n\n\n<p>Stay at dify\/docker, and start Dify:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# -d means let container run in the backend\ndocker compose up -d\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># -d means let container run in the backend<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">compose<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">up<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #569CD6\">-d<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"556\" height=\"188\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-4.png\" alt=\"\" class=\"wp-image-377\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-4.png 556w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/03\/image-4-300x101.png 300w\" sizes=\"auto, (max-width: 556px) 100vw, 556px\" \/><\/figure>\n\n\n\n<p>Now you can login Dify with <strong>https:\/\/ip:port<\/strong>. You need to set up admin account when first log in.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"481\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-1-1024x481.png\" alt=\"\" class=\"wp-image-383\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-1-1024x481.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-1-300x141.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-1-768x361.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-1-1536x721.png 1536w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-1.png 1833w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-install-and-config-xinference\">[OPTION 1] Install and config Xinference<\/h1>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"aioseo-install-xinference\">Install Xinference<\/h2>\n\n\n\n<p>Official guideline could be found <a href=\"https:\/\/inference.readthedocs.io\/en\/latest\/getting_started\/using_docker_image.html\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>. Recommend using Docker image. It is the easiest way to deploy, though it would takes some time to pull the image from dockerhub or Aliyun. <\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-style-default\"><img loading=\"lazy\" decoding=\"async\" width=\"750\" height=\"328\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-2.png\" alt=\"\" class=\"wp-image-397\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-2.png 750w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-2-300x131.png 300w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/figure>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(1 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# pull image from Dockerhub\ndocker pull xprobe\/xinference:latest\n\n# pull image from Aliyun\ndocker pull registry.cn-hangzhou.aliyuncs.com\/xprobe_xinference\/xinference:latest\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># pull image from Dockerhub<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">pull<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">xprobe\/xinference:latest<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># pull image from Aliyun<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">pull<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">registry.cn-hangzhou.aliyuncs.com\/xprobe_xinference\/xinference:latest<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<p>Choose the <em><strong>latest<\/strong><\/em> version should be fine. Usually I use gpu, so skip the option <strong><em>-cpu<\/em><\/strong>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"321\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-3-1024x321.png\" alt=\"\" class=\"wp-image-408\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-3-1024x321.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-3-300x94.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-3-768x241.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-3.png 1212w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"aioseo-start-xinference\">Start Xinference<\/h2>\n\n\n\n<p>Use Docker run command to start the container.<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(2 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"docker run -d \\\n  -v \/opt\/xinference:\/root\/.xinference \\\n  -v \/opt\/huggingface:\/root\/.cache\/huggingface \\\n  -v \/opt\/modelscope:\/root\/.cache\/modelscope \\\n  -p 9997:9997 \\\n  --gpus all \\\n  xprobe\/xinference:latest \\\n  xinference-local -H 0.0.0.0\n\n# Param explanation\ndocker run \\\n  # Run in backend\n  -d \\\n  # Home path, where the logs saved\n  -v \/your\/home\/path\/.xinference:\/root\/.xinference \\\n  # Your local model files, download from huggingface or modelscope\n  -v \/your\/huggingface\/path:\/root\/.cache\/huggingface \\\n  -v \/your\/modelscope\/path:\/root\/.cache\/modelscope \\\n  # &lt;HostPort&gt;:&lt;ContainerPort&gt;\n  -p 9997:9997 \\\n  # Access all gpus, or specific one or more gpus\n  --gpus all \\\n  # Docker image name and version\n  xprobe\/xinference:v&lt;version&gt;\\\n  # Can be accessed from external systems through &lt;IP&gt;:&lt;PORT&gt;\n  xinference-local -H 0.0.0.0\n  \" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">run<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #569CD6\">-d<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #569CD6\">-v<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">\/opt\/xinference:\/root\/.xinference<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #569CD6\">-v<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">\/opt\/huggingface:\/root\/.cache\/huggingface<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #569CD6\">-v<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">\/opt\/modelscope:\/root\/.cache\/modelscope<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #569CD6\">-p<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #B5CEA8\">9997<\/span><span style=\"color: #CE9178\">:9997<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #569CD6\">--gpus<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">all<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #CE9178\">xprobe\/xinference:latest<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #CE9178\">xinference-local<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #569CD6\">-H<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #B5CEA8\">0.0<\/span><span style=\"color: #CE9178\">.0.0<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># Param explanation<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">docker<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">run<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># Run in backend<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">-d<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># Home path, where the logs saved<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">-v<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">\/your\/home\/path\/.xinference:\/root\/.xinference<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># Your local model files, download from huggingface or modelscope<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">-v<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">\/your\/huggingface\/path:\/root\/.cache\/huggingface<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #569CD6\">-v<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">\/your\/modelscope\/path:\/root\/.cache\/modelscope<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># &lt;HostPort&gt;:&lt;ContainerPort&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">-p<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #B5CEA8\">9997<\/span><span style=\"color: #CE9178\">:9997<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># Access all gpus, or specific one or more gpus<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">--gpus<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">all<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #D7BA7D\">\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># Docker image name and version<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">xprobe\/xinference:v&lt;version&gt;\\<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #6A9955\"># Can be accessed from external systems through &lt;IP&gt;:&lt;PORT&gt;<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><span style=\"color: #DCDCAA\">xinference-local<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #569CD6\">-H<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #B5CEA8\">0.0<\/span><span style=\"color: #CE9178\">.0.0<\/span><\/span>\n<span class=\"line\"><span style=\"color: #D4D4D4\">  <\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"aioseo-register-model\">Register model<\/h2>\n\n\n\n<p>You can launch model online (it will download models directly). Or you can register your local models. Here I will show the second way.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"519\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-5-1024x519.png\" alt=\"\" class=\"wp-image-456\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-5-1024x519.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-5-300x152.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-5-768x389.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-5-1536x778.png 1536w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-5.png 1797w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Specifications of the models could be found <a href=\"https:\/\/inference.readthedocs.io\/en\/latest\/models\/builtin\/llm\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"927\" height=\"864\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-14.png\" alt=\"\" class=\"wp-image-494\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-14.png 927w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-14-300x280.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-14-768x716.png 768w\" sizes=\"auto, (max-width: 927px) 100vw, 927px\" \/><\/figure>\n\n\n\n<p>For example, I have downloaded below model files from <a href=\"https:\/\/huggingface.co\/\" target=\"_blank\" rel=\"noopener\" title=\"\">Huggingface<\/a> and uploaded to the server:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>(LLM) THUDM\/glm-4-9b-chat<\/li>\n\n\n\n<li>(Embedding) BAAI\/bge-large-zh-v1.5<\/li>\n\n\n\n<li>(Rerank) BAAI\/bge-reranker-large<\/li>\n<\/ul>\n\n\n\n<p>Register LLM:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"468\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-16-1024x468.png\" alt=\"\" class=\"wp-image-500\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-16-1024x468.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-16-300x137.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-16-768x351.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-16-1536x702.png 1536w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-16.png 1870w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"967\" height=\"386\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-17.png\" alt=\"\" class=\"wp-image-502\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-17.png 967w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-17-300x120.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-17-768x307.png 768w\" sizes=\"auto, (max-width: 967px) 100vw, 967px\" \/><\/figure>\n\n\n\n<p>Register embedding model:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"761\" height=\"581\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-18.png\" alt=\"\" class=\"wp-image-504\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-18.png 761w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-18-300x229.png 300w\" sizes=\"auto, (max-width: 761px) 100vw, 761px\" \/><\/figure>\n\n\n\n<p>Register rerank model:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"764\" height=\"463\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-19.png\" alt=\"\" class=\"wp-image-506\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-19.png 764w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-19-300x182.png 300w\" sizes=\"auto, (max-width: 764px) 100vw, 764px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"aioseo-launch-model\">Launch model<\/h2>\n\n\n\n<p>For example, launch LLM:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"512\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-20-1024x512.png\" alt=\"\" class=\"wp-image-508\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-20-1024x512.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-20-300x150.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-20-768x384.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-20.png 1157w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"506\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-21-1024x506.png\" alt=\"\" class=\"wp-image-509\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-21-1024x506.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-21-300x148.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-21-768x380.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-21-1536x759.png 1536w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-21.png 1867w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-option-2-install-and-config-ollama\">[OPTION 2] Install and config Ollama<\/h1>\n\n\n\n<p>Download guideline is <a href=\"https:\/\/ollama.com\/download\/linux\" target=\"_blank\" rel=\"noopener\" title=\"\">here<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"472\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-22-1024x472.png\" alt=\"\" class=\"wp-image-513\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-22-1024x472.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-22-300x138.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-22-768x354.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-22.png 1047w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Some frequently used command:<\/p>\n\n\n\n<div class=\"wp-block-kevinbatdorf-code-block-pro padding-bottom-disabled cbp-has-line-numbers\" data-code-block-pro-font-family=\"Code-Pro-JetBrains-Mono\" style=\"font-size:.875rem;font-family:Code-Pro-JetBrains-Mono,ui-monospace,SFMono-Regular,Menlo,Monaco,Consolas,monospace;--cbp-line-number-color:#D4D4D4;--cbp-line-number-width:calc(2 * 0.6 * .875rem);line-height:1.25rem;--cbp-tab-width:2;tab-size:var(--cbp-tab-width, 2)\"><span style=\"display:block;padding:16px 0 0 16px;margin-bottom:-1px;width:100%;text-align:left;background-color:#1E1E1E\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"54\" height=\"14\" viewBox=\"0 0 54 14\"><g fill=\"none\" fill-rule=\"evenodd\" transform=\"translate(1 1)\"><circle cx=\"6\" cy=\"6\" r=\"6\" fill=\"#FF5F56\" stroke=\"#E0443E\" stroke-width=\".5\"><\/circle><circle cx=\"26\" cy=\"6\" r=\"6\" fill=\"#FFBD2E\" stroke=\"#DEA123\" stroke-width=\".5\"><\/circle><circle cx=\"46\" cy=\"6\" r=\"6\" fill=\"#27C93F\" stroke=\"#1AAB29\" stroke-width=\".5\"><\/circle><\/g><\/svg><\/span><span role=\"button\" tabindex=\"0\" data-code=\"# run ollama in backend\nollama serve\n\n# show model list\nollama list\n\n# show running models\nollama ps\n\n# pull model\nollama pull [ModelName]\" style=\"color:#D4D4D4;display:none\" aria-label=\"Copy\" class=\"code-block-pro-copy-button\"><svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" style=\"width:24px;height:24px\" fill=\"none\" viewBox=\"0 0 24 24\" stroke=\"currentColor\" stroke-width=\"2\"><path class=\"with-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2m-6 9l2 2 4-4\"><\/path><path class=\"without-check\" stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M9 5H7a2 2 0 00-2 2v12a2 2 0 002 2h10a2 2 0 002-2V7a2 2 0 00-2-2h-2M9 5a2 2 0 002 2h2a2 2 0 002-2M9 5a2 2 0 012-2h2a2 2 0 012 2\"><\/path><\/svg><\/span><pre class=\"shiki dark-plus\" style=\"background-color: #1E1E1E\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color: #6A9955\"># run ollama in backend<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">ollama<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">serve<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># show model list<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">ollama<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">list<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># show running models<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">ollama<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">ps<\/span><\/span>\n<span class=\"line\"><\/span>\n<span class=\"line\"><span style=\"color: #6A9955\"># pull model<\/span><\/span>\n<span class=\"line\"><span style=\"color: #DCDCAA\">ollama<\/span><span style=\"color: #D4D4D4\"> <\/span><span style=\"color: #CE9178\">pull<\/span><span style=\"color: #D4D4D4\"> [ModelName]<\/span><\/span><\/code><\/pre><span style=\"display:flex;align-items:flex-end;padding:10px;width:100%;justify-content:flex-end;background-color:#1E1E1E;color:#c7c7c7;font-size:12px;line-height:1;position:relative\">Bash<\/span><\/div>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-integrate-xinference-in-dify\">Integrate Xinference\/Ollama in Dify<\/h1>\n\n\n\n<p>Login Dify, go to Settings &#8211; Model Provider, then install Ollama :<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"238\" height=\"443\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-23.png\" alt=\"\" class=\"wp-image-516\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-23.png 238w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-23-161x300.png 161w\" sizes=\"auto, (max-width: 238px) 100vw, 238px\" \/><\/figure>\n\n\n\n<p>Add your model here, then able to use it in Dify workflow:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"327\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-25-1024x327.png\" alt=\"\" class=\"wp-image-527\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-25-1024x327.png 1024w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-25-300x96.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-25-768x246.png 768w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-25.png 1107w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1009\" height=\"901\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-26.png\" alt=\"\" class=\"wp-image-535\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-26.png 1009w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-26-300x268.png 300w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-26-768x686.png 768w\" sizes=\"auto, (max-width: 1009px) 100vw, 1009px\" \/><\/figure>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"aioseo-integrate-ollama-in-dify\">FAQ<\/h1>\n\n\n\n<div data-schema-only=\"false\" class=\"wp-block-aioseo-faq\" id=\"aioseo-already-installed-docker-and-docker-compose-but-systemctl-status-docker-got-error\"><h3 class=\"aioseo-faq-block-question\">Already installed Docker and Docker Compose, but systemctl status docker got error.<\/h3><div class=\"aioseo-faq-block-answer\">\n<p>It might be due to a version mismatch between the Docker client and Docker CE.<br>Follow the <a href=\"https:\/\/docs.docker.com\/engine\/install\/ubuntu\/\" target=\"_blank\" rel=\"noopener\" title=\"\">Docker&#8217;s guideline<\/a> to reinstall.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"714\" height=\"583\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-29.png\" alt=\"\" class=\"wp-image-547\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-29.png 714w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-29-300x245.png 300w\" sizes=\"auto, (max-width: 714px) 100vw, 714px\" \/><\/figure>\n\n\n\n<p><\/p>\n<\/div><\/div>\n\n\n\n<div data-schema-only=\"false\" class=\"wp-block-aioseo-faq\" id=\"aioseo-could-not-register\"><h3 class=\"aioseo-faq-block-question\">Could not find model when register model.<\/h3><div class=\"aioseo-faq-block-answer\">\n<p>The model path should be the directory of your container, not the one of your host.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"629\" height=\"271\" src=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-30.png\" alt=\"\" class=\"wp-image-552\" srcset=\"https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-30.png 629w, https:\/\/home.miawy.com\/wp-content\/uploads\/2025\/04\/image-30-300x129.png 300w\" sizes=\"auto, (max-width: 629px) 100vw, 629px\" \/><\/figure>\n\n\n\n<p><\/p>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"<p>This tutorial will show how to build your own RAG: deploy Dify + Xinference\/Ollama in local server. What is Dify? Check Environment Check Docker and Docker&#46;&#46;&#46;<\/p>\n","protected":false},"author":1,"featured_media":282,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[17],"tags":[],"class_list":["post-277","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/posts\/277","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/home.miawy.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=277"}],"version-history":[{"count":241,"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/posts\/277\/revisions"}],"predecessor-version":[{"id":603,"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/posts\/277\/revisions\/603"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/home.miawy.com\/index.php?rest_route=\/wp\/v2\/media\/282"}],"wp:attachment":[{"href":"https:\/\/home.miawy.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=277"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/home.miawy.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=277"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/home.miawy.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=277"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}