File tree Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Expand file tree Collapse file tree 2 files changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -19,7 +19,7 @@ Let's think about this step by step.<|im_end|>
1919
2020## Answer Extraction
2121``` python
22- # Remove reasoning, extract final answer
22+ # Extract final answer
2323output = full_output.split(' <|/thinking|>' )[- 1 ] if ' <|/thinking|>' in full_output else full_output
2424answer = re.search(r ' ####\s * ( \d + ) ' , output)
2525final_answer = answer.group(1 ) if answer else output.strip()
Original file line number Diff line number Diff line change 22
33## Tokenization
44``` python
5- tokenizer = AutoTokenizer.from_pretrained(model_path) # Use model's tokenizer
5+ tokenizer = AutoTokenizer.from_pretrained(model_path)
66max_length = 2048
77```
88
@@ -20,6 +20,6 @@ Summarize this article:
2020
2121## Verification
2222``` bash
23- # Check first token is 128000 (begin_of_text)
23+ # Verify first token
2424python -c " print(tokenizer.encode('<|begin_of_text|>')[0])"
2525```
You can’t perform that action at this time.
0 commit comments