{"id":13657,"date":"2024-11-20T14:03:23","date_gmt":"2024-11-20T06:03:23","guid":{"rendered":"https:\/\/www.aisharenet.com\/?p=13657"},"modified":"2024-11-27T16:30:38","modified_gmt":"2024-11-27T08:30:38","slug":"petals","status":"publish","type":"post","link":"https:\/\/www.kdjingpai.com\/ja\/petals\/","title":{"rendered":"Petals\uff1a\u5206\u5e03\u5f0f\u5171\u4eabGPU\u8fd0\u884c\u548c\u5fae\u8c03\u5927\u8bed\u8a00\u6a21\u578b\uff0c\u50cfBitTorrent\u7f51\u7edc\u4e00\u6837\u5171\u4eabGPU\u8d44\u6e90"},"content":{"rendered":"<p>Petals \u662f\u4e00\u4e2a\u7531 BigScience Workshop \u5f00\u53d1\u7684\u5f00\u6e90\u9879\u76ee\uff0c\u65e8\u5728\u901a\u8fc7\u5206\u5e03\u5f0f\u8ba1\u7b97\u7684\u65b9\u5f0f\u8fd0\u884c\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff08LLMs\uff09\u3002\u7528\u6237\u53ef\u4ee5\u5728\u5bb6\u4e2d\u4f7f\u7528\u6d88\u8d39\u7ea7 GPU \u6216 Google Colab \u8fd0\u884c\u548c\u5fae\u8c03 LLMs\uff0c\u5982 Llama 3.1\u3001Mixtral\u3001Falcon \u548c BLOOM\u3002Petals \u91c7\u7528\u7c7b\u4f3c BitTorrent \u7684\u65b9\u5f0f\uff0c\u5c06\u6a21\u578b\u7684\u4e0d\u540c\u90e8\u5206\u5206\u5e03\u5728\u591a\u4e2a\u7528\u6237\u7684\u8bbe\u5907\u4e0a\uff0c\u4ece\u800c\u5b9e\u73b0\u9ad8\u6548\u7684\u63a8\u7406\u548c\u5fae\u8c03\u3002<\/p>\n<p><img decoding=\"async\" title=\"Petals\uff1a\u5206\u5e03\u5f0f\u63a8\u7406\u8fd0\u884c\u3001\u5fae\u8c03\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff0c\u50cfBitTorrent\u7f51\u7edc\u4e00\u6837\u5171\u4eabGPU\u8d44\u6e90-1\" src=\"https:\/\/www.kdjingpai.com\/wp-content\/uploads\/2024\/11\/fc497e4dacab578.jpg\" alt=\"Petals\uff1a\u5206\u5e03\u5f0f\u63a8\u7406\u8fd0\u884c\u3001\u5fae\u8c03\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff0c\u50cfBitTorrent\u7f51\u7edc\u4e00\u6837\u5171\u4eabGPU\u8d44\u6e90-1\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2>\u529f\u80fd\u5217\u8868<\/h2>\n<ul>\n<li><strong>\u8fd0\u884c\u5927\u578b\u8bed\u8a00\u6a21\u578b<\/strong>\uff1a\u652f\u6301 Llama 3.1\uff08\u6700\u9ad8 405B\uff09\u3001Mixtral\uff088x22B\uff09\u3001Falcon\uff0840B+\uff09\u548c BLOOM\uff08176B\uff09\u7b49\u6a21\u578b\u3002<\/li>\n<li><strong>\u5206\u5e03\u5f0f\u63a8\u7406<\/strong>\uff1a\u901a\u8fc7\u5206\u5e03\u5f0f\u7f51\u7edc\u8fd0\u884c\u6a21\u578b\uff0c\u5355\u6279\u6b21\u63a8\u7406\u901f\u5ea6\u53ef\u8fbe 6 tokens\/sec\uff08Llama 2 70B\uff09\u548c 4 tokens\/sec\uff08Falcon 180B\uff09\u3002<\/li>\n<li><strong>\u5feb\u901f\u5fae\u8c03<\/strong>\uff1a\u652f\u6301\u7528\u6237\u5bf9\u6a21\u578b\u8fdb\u884c\u5feb\u901f\u5fae\u8c03\uff0c\u9002\u7528\u4e8e\u5404\u79cd\u4efb\u52a1\u3002<\/li>\n<li><strong>\u793e\u533a\u9a71\u52a8<\/strong>\uff1a\u4f9d\u8d56\u793e\u533a\u7528\u6237\u5171\u4eab GPU \u8d44\u6e90\uff0c\u7528\u6237\u53ef\u4ee5\u8d21\u732e\u81ea\u5df1\u7684 GPU \u6765\u589e\u52a0 Petals \u7684\u8ba1\u7b97\u80fd\u529b\u3002<\/li>\n<li><strong>\u7075\u6d3b\u7684 API<\/strong>\uff1a\u63d0\u4f9b\u7c7b\u4f3c PyTorch \u548c Transformers \u7684\u7075\u6d3b API\uff0c\u652f\u6301\u81ea\u5b9a\u4e49\u8def\u5f84\u548c\u67e5\u770b\u9690\u85cf\u72b6\u6001\u3002<\/li>\n<li><strong>\u9690\u79c1\u4fdd\u62a4<\/strong>\uff1a\u6570\u636e\u5904\u7406\u901a\u8fc7\u516c\u5171\u7f51\u7edc\u8fdb\u884c\uff0c\u7528\u6237\u53ef\u4ee5\u8bbe\u7f6e\u79c1\u6709\u7f51\u7edc\u4ee5\u4fdd\u62a4\u654f\u611f\u6570\u636e\u3002<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>\u4f7f\u7528\u5e2e\u52a9<\/h2>\n<h3>\u5b89\u88c5\u4e0e\u4f7f\u7528<\/h3>\n<ol>\n<li><strong>\u5b89\u88c5\u4f9d\u8d56<\/strong>\uff1a\n<ul>\n<li><strong>Linux + Anaconda<\/strong>\uff1a\n<pre><code>conda install pytorch pytorch-cuda=11.7 -c pytorch -c nvidia\r\npip install git+https:\/\/github.com\/bigscience-workshop\/petals\r\npython -m petals.cli.run_server meta-llama\/Meta-Llama-3.1-405B-Instruct\r\n<\/code><\/pre>\n<\/li>\n<li><strong>Windows + WSL<\/strong>\uff1a\u8bf7\u53c2\u8003\u00a0<a href=\"https:\/\/github.com\/bigscience-workshop\/petals\/wiki\">Wiki<\/a>\u3002<\/li>\n<li><strong>Docker<\/strong>\uff1a\n<pre><code>sudo docker run -p 31330:31330 --ipc host --gpus all --volume petals-cache:\/cache --rm \\\r\nlearningathome\/petals:main \\\r\npython -m petals.cli.run_server --port 31330 meta-llama\/Meta-Llama-3.1-405B-Instruct\r\n<\/code><\/pre>\n<\/li>\n<li><strong>macOS + Apple M1\/M2 GPU<\/strong>\uff1a\n<pre><code>brew install python\r\npython3 -m pip install git+https:\/\/github.com\/bigscience-workshop\/petals\r\npython3 -m petals.cli.run_server meta-llama\/Meta-Llama-3.1-405B-Instruct\r\n<\/code><\/pre>\n<\/li>\n<\/ul>\n<\/li>\n<li><strong>\u8fd0\u884c\u6a21\u578b<\/strong>\uff1a\n<ul>\n<li>\u9009\u62e9\u4efb\u610f\u53ef\u7528\u6a21\u578b\uff0c\u4f8b\u5982\uff1a\n<pre><code>from transformers import AutoTokenizer\r\nfrom petals import AutoDistributedModelForCausalLM\r\nmodel_name = \"meta-llama\/Meta-Llama-3.1-405B-Instruct\"\r\ntokenizer = AutoTokenizer.from_pretrained(model_name)\r\nmodel = AutoDistributedModelForCausalLM.from_pretrained(model_name)\r\ninputs = tokenizer(\"A cat sat\", return_tensors=\"pt\")[\"input_ids\"]\r\noutputs = model.generate(inputs, max_new_tokens=5)\r\nprint(tokenizer.decode(outputs[0]))\r\n<\/code><\/pre>\n<\/li>\n<\/ul>\n<\/li>\n<li><strong>\u8d21\u732e GPU<\/strong>\uff1a\n<ul>\n<li>\u7528\u6237\u53ef\u4ee5\u901a\u8fc7\u8fde\u63a5 GPU \u6765\u589e\u52a0 Petals \u7684\u8ba1\u7b97\u80fd\u529b\uff0c\u5177\u4f53\u64cd\u4f5c\u8bf7\u53c2\u8003\u00a0<a href=\"https:\/\/github.com\/bigscience-workshop\/petals\">Model Hub<\/a>\u3002<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<h3>\u4e3b\u8981\u529f\u80fd\u64cd\u4f5c\u6d41\u7a0b<\/h3>\n<ol>\n<li><strong>\u9009\u62e9\u6a21\u578b<\/strong>\uff1a\u8bbf\u95ee\u00a0<a href=\"https:\/\/petals.dev\/\">Petals<\/a>\u00a0\u7f51\u7ad9\uff0c\u9009\u62e9\u6240\u9700\u7684\u6a21\u578b\u3002<\/li>\n<li><strong>\u52a0\u8f7d\u6a21\u578b<\/strong>\uff1a\u6839\u636e\u4e0a\u8ff0\u5b89\u88c5\u6b65\u9aa4\uff0c\u52a0\u8f7d\u5e76\u8fd0\u884c\u6a21\u578b\u3002<\/li>\n<li><strong>\u5fae\u8c03\u6a21\u578b<\/strong>\uff1a\u4f7f\u7528 Petals \u63d0\u4f9b\u7684 API \u5bf9\u6a21\u578b\u8fdb\u884c\u5fae\u8c03\uff0c\u9002\u7528\u4e8e\u5404\u79cd\u4efb\u52a1\u3002<\/li>\n<li><strong>\u751f\u6210\u6587\u672c<\/strong>\uff1a\u901a\u8fc7\u5206\u5e03\u5f0f\u7f51\u7edc\u751f\u6210\u6587\u672c\uff0c\u9002\u7528\u4e8e\u804a\u5929\u673a\u5668\u4eba\u548c\u4ea4\u4e92\u5f0f\u5e94\u7528\u3002<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Petals \u662f\u4e00\u4e2a\u7531 BigScience Workshop \u5f00\u53d1\u7684\u5f00\u6e90\u9879\u76ee\uff0c\u65e8\u5728\u901a\u8fc7\u5206\u5e03\u5f0f\u8ba1\u7b97\u7684\u65b9\u5f0f\u8fd0\u884c\u5927\u578b\u8bed\u8a00\u6a21\u578b\uff08LLMs\uff09\u3002\u7528\u6237\u53ef\u4ee5\u5728\u5bb6\u4e2d\u4f7f\u7528\u6d88\u8d39\u7ea7 GPU \u6216 Google Colab \u8fd0\u884c\u548c\u5fae\u8c03 LLMs\uff0c\u5982 Llama 3&#8230;.<\/p>\n","protected":false},"author":1,"featured_media":61273,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,232],"class_list":["post-13657","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tool","tag-damoxingweidiao","tag-bendebushukaiyuanba"],"_links":{"self":[{"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/posts\/13657","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/comments?post=13657"}],"version-history":[{"count":0,"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/posts\/13657\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/media\/61273"}],"wp:attachment":[{"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/media?parent=13657"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/categories?post=13657"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kdjingpai.com\/ja\/wp-json\/wp\/v2\/tags?post=13657"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}