bug: ParseExtractionResponse fails when LLM outputs quality_flags as string instead of array

[mpeter]

Summary

dewey curate fails with json: cannot unmarshal string into Go struct field KnowledgeFile.quality_flags of type curate.QualityFlag when the LLM outputs "quality_flags": "none" or "quality_flags": "[]" (string) instead of "quality_flags": [] (JSON array).

Reproduction

Run dewey curate against any store. The extraction prompt instructs the LLM to produce quality_flags: [] for items with no issues, but Claude (and likely other LLMs) sometimes outputs a string value instead:

{
  "tag": "account-ownership",
  "category": "context",
  "quality_flags": "none",
  ...
}

ParseExtractionResponse does strict json.Unmarshal into []KnowledgeFile, which fails on the type mismatch.

Root Cause

curate/curate.go:

var files []KnowledgeFile
if err := json.Unmarshal([]byte(cleaned), &files); err != nil {
    return nil, fmt.Errorf("parse LLM response as JSON array: %w", err)
}

LLM output is non-deterministic. The strict unmarshal rejects any item where quality_flags is a JSON string, discarding the entire response even when all other items are valid.

Fix

Parse into []json.RawMessage first, then unmarshal each item individually. When strict unmarshal fails, fall back to a permissive struct that accepts quality_flags as json.RawMessage and discards non-array values:

var rawFiles []json.RawMessage
if err := json.Unmarshal([]byte(cleaned), &rawFiles); err != nil {
    return nil, fmt.Errorf("parse LLM response as JSON array: %w", err)
}

for _, raw := range rawFiles {
    var f KnowledgeFile
    if err := json.Unmarshal(raw, &f); err != nil {
        // Fall back: accept quality_flags as any type, discard non-array
        // ... (see PR for full implementation)
    }
    files = append(files, f)
}

This makes the parser tolerant of LLM type inconsistencies while preserving all valid knowledge items.

Additional context

The extraction prompt could also be improved to more firmly instruct the LLM to use [] (not a string), but prompt engineering alone is not a reliable fix for type safety — the parser should be defensive.