diff --git a/Online/README.md b/Online/README.md index 14cf136..94cb54b 100644 --- a/Online/README.md +++ b/Online/README.md @@ -76,6 +76,7 @@ | [VideoClassification](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/VideoClassification) | 8.0.0.beta1 | 2.6.0 |8T16G | | [MaskGeneration](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/MaskGeneration) | 8.1.RC1 | 2.6.0 | 8T16G | | [DocumentQuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DocumentQuestionAnswering) | 8.0.0.beta1 | 2.6.0 | 20T24G | +| [QuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/QuestionAnswering) | 8.1.RC1 | 2.6.0 | 20T24G | diff --git a/Online/community/QuestionAnswering/Question_Answering.ipynb b/Online/community/QuestionAnswering/Question_Answering.ipynb new file mode 100644 index 0000000..76831bd --- /dev/null +++ b/Online/community/QuestionAnswering/Question_Answering.ipynb @@ -0,0 +1,434 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "77a5a292", + "metadata": {}, + "source": [ + "# 基于 Qwen1.5-0.5b 的Question Answering系统\n", + "## 项目介绍\n", + "本项目旨在利用Qwen1.5-0.5b大语言模型，构建一个智能问答系统。​该系统不仅展示了大语言模型在专业领域文档理解和信息提取方面的能力，同时也为开发者提供了一个实践范例，说明如何结合预训练模型与特定领域知识，打造高效的问答解决方案。" + ] + }, + { + "cell_type": "markdown", + "id": "07953402-dbd0-45f3-a297-e5c02df03429", + "metadata": { + "tags": [] + }, + "source": [ + "## 环境准备\n", + "开发者拿到香橙派开发板后，首先需要进行硬件资源确认，镜像烧录及CANN和MindSpore版本的升级，才可运行该案例，具体如下：\n", + "\n", + "开发板：香橙派AIpro 24G 20T开发板 \n", + "开发板镜像: Ubuntu镜像 \n", + "CANN Toolkit/Kernels：8.1.RC1 \n", + "MindSpore: 2.6.0 \n", + "MindSpore NLP: 0.4.1 \n", + "Python: 3.9\n", + "\n", + "### 镜像烧录\n", + "运行该案例需要烧录香橙派官网ubuntu镜像，烧录流程参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--镜像烧录](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/orange_pi/environment_setup.html) 章节。\n", + "\n", + "### CANN升级\n", + "CANN升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--CANN升级](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/orange_pi/environment_setup.html)章节。\n", + "\n", + "### MindSpore升级\n", + "MindSpore升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--MindSpore升级](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/orange_pi/environment_setup.html)章节。" + ] + }, + { + "cell_type": "markdown", + "id": "d0f43f46", + "metadata": {}, + "source": [ + "## 安装依赖包\n", + "本项目主要基于 MindSpore NLP 和 Mindspore 进行开发，除上述两个库外，需先安装相关依赖库\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ac97bb42", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "!pip install jieba==0.42.1" + ] + }, + { + "cell_type": "markdown", + "id": "15d6f05c", + "metadata": {}, + "source": [ + "## 加载Qwen1.5-0.5b模型\n", + "本项目使用小规模蒸馏模型\n", + "\n", + "基于Mindspore框架和MindSpore NLP库开发安装还是很方便的" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "62545fef-589f-4632-8922-5199ae135300", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/HwHiAiUser/.local/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/home/HwHiAiUser/.local/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n", + "/home/HwHiAiUser/.local/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/home/HwHiAiUser/.local/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n", + "[WARNING] ME(7230:255085986443296,MainProcess):2025-09-11-15:30:28.629.326 [mindspore/context.py:1402] For 'context.set_context', the parameter 'ascend_config' will be deprecated and removed in a future version. Please use the api mindspore.device_context.ascend.op_precision.precision_mode(),\n", + " mindspore.device_context.ascend.op_precision.op_precision_mode(),\n", + " mindspore.device_context.ascend.op_precision.matmul_allow_hf32(),\n", + " mindspore.device_context.ascend.op_precision.conv_allow_hf32(),\n", + " mindspore.device_context.ascend.op_tuning.op_compile() instead.\n", + "/home/HwHiAiUser/.local/lib/python3.9/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", + " from .autonotebook import tqdm as notebook_tqdm\n", + "Building prefix dict from the default dictionary ...\n", + "Loading model from cache /tmp/jieba.cache\n", + "Loading model cost 1.471 seconds.\n", + "Prefix dict has been built successfully.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "正在加载模型...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Qwen2ForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly overwritten. However, it doesn't directly inherit from `GenerationMixin`.`PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.\n", + " - If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).\n", + " - If you are not the owner of the model architecture class, please contact the model code owner to update it.\n", + "Sliding Window Attention is enabled but not implemented for `eager`; unexpected results may be encountered.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "模型加载完成！\n" + ] + } + ], + "source": [ + "import mindspore\n", + "from mindnlp.transformers import AutoModelForCausalLM, AutoTokenizer\n", + "from mindnlp.transformers import TextIteratorStreamer\n", + "from threading import Thread\n", + "\n", + "# 加载tokenizer和模型\n", + "print(\"正在加载模型...\")\n", + "tokenizer = AutoTokenizer.from_pretrained(\"/home/HwHiAiUser/Qwen1.5-0.5B-Chat\", mirror=\"modelers\", ms_dtype=mindspore.float16)\n", + "model = AutoModelForCausalLM.from_pretrained(\"/home/HwHiAiUser/Qwen1.5-0.5B-Chat\", mirror=\"modelers\", ms_dtype=mindspore.float16)\n", + "print(\"模型加载完成！\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "20ecde6c", + "metadata": {}, + "source": [ + "## 构建回答的流式输出函数" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "bdb00370-ac2b-4e7e-9cc9-3130a535ad2b", + "metadata": {}, + "outputs": [], + "source": [ + "system_prompt = \"你是个能回答问题的专家，需要根据用户的需求来具体解答\"\n", + "def chat_with_model(user_input):\n", + " \"\"\"与模型进行单轮对话，支持流式输出\"\"\"\n", + " # 构建消息格式\n", + " messages = [\n", + " {'role': 'system', 'content': system_prompt},\n", + " {'role': 'user', 'content': user_input}\n", + " ]\n", + " \n", + " # 应用聊天模板\n", + " input_ids = tokenizer.apply_chat_template(\n", + " messages,\n", + " add_generation_prompt=True,\n", + " return_tensors=\"ms\",\n", + " tokenize=True\n", + " )\n", + " \n", + " # 创建流式输出器\n", + " streamer = TextIteratorStreamer(tokenizer, timeout=300, skip_prompt=True, skip_special_tokens=True)\n", + " \n", + " # 生成参数\n", + " generate_kwargs = dict(\n", + " input_ids=input_ids,\n", + " streamer=streamer,\n", + " max_new_tokens=1024,\n", + " do_sample=True,\n", + " top_p=0.9,\n", + " temperature=0.1,\n", + " num_beams=1,\n", + " pad_token_id=tokenizer.eos_token_id\n", + " )\n", + " \n", + " # 在单独线程中启动生成\n", + " t = Thread(target=model.generate, kwargs=generate_kwargs)\n", + " t.start()\n", + " \n", + " # 流式输出tokens\n", + " full_response = \"\"\n", + " for new_token in streamer:\n", + " if '' in new_token: # 检查停止token\n", + " break\n", + " print(new_token, end=\"\", flush=True)\n", + " full_response += new_token\n", + " \n", + " return \"回答完毕\"" + ] + }, + { + "cell_type": "markdown", + "id": "4e6affcd-f9fd-48c4-a5bf-d4ddbbc23b03", + "metadata": {}, + "source": [ + "## 单轮LLM问答" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "a2e6015e-4235-4013-9645-e64fee22f227", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "我是来自阿里云的大规模语言模型，我叫通义千问。" + ] + }, + { + "data": { + "text/plain": [ + "'回答完毕'" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "query=\"你是谁\"\n", + "chat_with_model(f\"请回答：{query}\")" + ] + }, + { + "cell_type": "markdown", + "id": "55da8aa1-a41b-429f-9466-6da945406253", + "metadata": {}, + "source": [ + "## 连续多轮LLM问答" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "582b86b4-ef3e-481f-8b87-d132b1fc9d75", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "=== 连续问答已启动（输入 q/quit/exit 退出）===\n", + "\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "你： 你是谁\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "助手：我是来自阿里云的大规模语言模型，我叫通义千问。\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "你： 介绍一下华为的主要业务\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "助手：华为是一家中国领先的通信设备和解决方案提供商。其主要业务包括智能手机、互联网设备（如路由器、交换机）、云计算服务以及大数据分析等。同时，华为还提供数据存储、网络安全、人工智能等领域的技术解决方案。\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "你： q\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[结束对话]\n" + ] + } + ], + "source": [ + "# 直接运行本单元格，然后在提示符连续输入问题；输入 q/quit/exit 结束。\n", + "\n", + "from mindnlp.transformers import TextIteratorStreamer\n", + "from threading import Thread\n", + "\n", + "# 基础生成参数（可保留默认值）\n", + "GEN_MAX_NEW_TOKENS = 512\n", + "GEN_TEMPERATURE = 0.7\n", + "GEN_TOP_P = 0.9\n", + "\n", + "# 可选的系统提示词（如果不需要，置为空字符串）\n", + "SYSTEM_PROMPT = \"你是一个专业的中文问答助手，请结合上下文，给出准确、简洁、有条理的回答。\"\n", + "\n", + "# 校验必要对象\n", + "if \"tokenizer\" not in globals() or \"model\" not in globals():\n", + " raise RuntimeError(\"未检测到 tokenizer 或 model，请先运行上文加载模型的单元格。\")\n", + "\n", + "def generate_once(messages):\n", + " \"\"\"\n", + " 基于 MindNLP 的聊天模板与流式生成，输入完整 messages（包含历史），返回本轮回复字符串。\n", + " messages: List[{'role': 'system'|'user'|'assistant', 'content': str}]\n", + " \"\"\"\n", + " # 1) 构造输入（MindNLP: return_tensors='ms'）\n", + " input_ids = tokenizer.apply_chat_template(\n", + " messages,\n", + " add_generation_prompt=True,\n", + " return_tensors=\"ms\",\n", + " tokenize=True\n", + " )\n", + "\n", + " # 2) 创建流式输出器（逐 token 返回字符串）\n", + " streamer = TextIteratorStreamer(\n", + " tokenizer=tokenizer,\n", + " skip_prompt=True,\n", + " skip_special_tokens=True\n", + " )\n", + "\n", + " # 3) 组装生成参数\n", + " generate_kwargs = dict(\n", + " input_ids=input_ids,\n", + " max_new_tokens=GEN_MAX_NEW_TOKENS,\n", + " do_sample=True,\n", + " temperature=GEN_TEMPERATURE,\n", + " top_p=GEN_TOP_P,\n", + " streamer=streamer,\n", + " eos_token_id=getattr(tokenizer, \"eos_token_id\", None),\n", + " pad_token_id=getattr(tokenizer, \"pad_token_id\", None),\n", + " )\n", + "\n", + " # 4) 在线程中启动生成（主线程消费 streamer）\n", + " t = Thread(target=model.generate, kwargs=generate_kwargs)\n", + " t.start()\n", + "\n", + " # 5) 流式打印并累计输出\n", + " full_text = \"\"\n", + " for piece in streamer:\n", + " print(piece, end=\"\", flush=True)\n", + " full_text += piece\n", + " print() # 换行\n", + "\n", + " return full_text.strip()\n", + "\n", + "# ========== 多轮对话主循环 ==========\n", + "history = []\n", + "if SYSTEM_PROMPT:\n", + " history.append({\"role\": \"system\", \"content\": SYSTEM_PROMPT})\n", + "\n", + "print(\"=== 连续问答已启动（输入 q/quit/exit 退出）===\\n\")\n", + "while True:\n", + " try:\n", + " user_text = input(\"你：\").strip()\n", + " except EOFError:\n", + " print(\"\\n[结束对话]\")\n", + " break\n", + "\n", + " if not user_text:\n", + " continue\n", + " if user_text.lower() in {\"q\", \"quit\", \"exit\"}:\n", + " print(\"[结束对话]\")\n", + " break\n", + "\n", + " # 追加用户消息并生成\n", + " history.append({\"role\": \"user\", \"content\": user_text})\n", + " print(\"助手：\", end=\"\", flush=True)\n", + " try:\n", + " answer = generate_once(history)\n", + " except Exception as e:\n", + " print(f\"[生成出错] {e}\")\n", + " # 出错时回滚上一轮的 user 消息，继续循环\n", + " history.pop()\n", + " continue\n", + "\n", + " # 记录助手消息，维护上下文\n", + " history.append({\"role\": \"assistant\", \"content\": answer})\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4e781c15-5224-4d91-82c9-368c4dce48ff", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "py39", + "language": "python", + "name": "py39" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.23" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/Online/community/QuestionAnswering/README.md b/Online/community/QuestionAnswering/README.md new file mode 100644 index 0000000..a2c5560 --- /dev/null +++ b/Online/community/QuestionAnswering/README.md @@ -0,0 +1,53 @@ +# Question Answering + +基于MindSpore框架和Qwen1.5-0.5b模型实现自动问答系统 + +## 介绍 + +基于香橙派AIPro，利用大语言模型，构建一个智能连续问答系统。 + +### 环境准备 + +开发者拿到香橙派开发板后，首先需要进行硬件资源确认，镜像烧录及CANN和MindSpore版本的升级，才可运行该案例，具体如下： + +开发板：香橙派Aipro或其他同硬件开发板 +开发板镜像: Ubuntu镜像 +`CANN Toolkit/Kernels：8.1RC1` +`MindSpore: 2.6.0` +`MindSpore NLP: 0.4.1` +`Python: 3.9` + +#### 镜像烧录 + +运行该案例需要烧录香橙派官网ubuntu镜像，烧录流程参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--镜像烧录](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/orange_pi/environment_setup.html) 章节。 + +#### CANN升级 + +CANN升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--CANN升级](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/orange_pi/environment_setup.html)章节。 + +#### MindSpore升级 + +MindSpore升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--MindSpore升级](https://www.mindspore.cn/tutorials/zh-CN/r2.7.0rc1/orange_pi/environment_setup.html)章节。 + +### 核心库版本 +``` +Python == 3.9 +MindSpore == 2.6.0 +mindnlp == 0.4.1 +sympy == 1.14.0 +jieba == 0.42.1 +tokenizers == 0.21.4 +``` +## 快速使用 + +建议下载qwen1.5-0.5b模型至本地路径（如/home/HwHiAiUser），然后修改ipynb中的模型路径参数： + +``` +git lfs install +git clone https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat +``` + +## 预期输出 + +模型根据用户键入内容回答问题。 + diff --git a/Online/community/README.md b/Online/community/README.md index 1230890..4e6ac9d 100644 --- a/Online/community/README.md +++ b/Online/community/README.md @@ -17,4 +17,5 @@ | [VideoClassification](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/VideoClassification) | 8.0.0.beta1 | 2.6.0 |8T16G | | [MaskGeneration](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/MaskGeneration) | 8.1.RC1 | 2.6.0 | 8T16G | | [DocumentQuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DocumentQuestionAnswering) | 8.0.0.beta1 | 2.6.0 | 20T24G | +| [QuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/QuestionAnswering) | 8.1.RC1 | 2.6.0 | 20T24G | diff --git a/README.md b/README.md index 7dc424b..1b5eaa0 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,7 @@ | [VideoClassification](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/VideoClassification) | 推理 | 8.0.0.beta1 | 2.6.0 |8T16G | | [MaskGeneration](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/MaskGeneration) | 推理 | 8.1.RC1 | 2.6.0 | 8T16G | | [DocumentQuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DocumentQuestionAnswering) | 推理 | 8.0.0.beta1 | 2.6.0 | 20T24G | +| [QuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/QuestionAnswering) | 推理 | 8.1.RC1 | 2.6.0 | 20T24G | > 注：在线案例指导请参考Online文件夹中的README文档