Date: 2026-04-09 Audience: 3-person BDH team preparing for science expo judging tomorrow Goal: Equip your team to answer high-pressure judge questions with technical depth, honesty, and confidence.
- Read Sections 2-4 together as a team (shared understanding).
- Split Sections 6-9 by role (one person leads each domain).
- Rehearse with Section 10 question bank (2 full mock rounds).
- Run one live demo using Section 11 and validate fallback plan.
- Memorize Section 12 "safe claims only" list.
If you only have 45 minutes, read these first:
- Section 2 (30-second + 2-minute pitch)
- Section 5 (codebase map)
- Section 7 (judge scoring logic)
- Section 10 (top 20 judge questions)
- Section 12 (what NOT to claim)
"We are building and evaluating BDH, a brain-inspired language model architecture that uses linear attention and synaptic-style memory mechanisms. Our project explores whether multi-scale memory traces, practical training pipelines, and distillation workflows can improve long-context behavior while keeping the system interpretable and deployable. We built training, benchmarking, distributed logits generation, and a live web comparison app to test this end to end."
"Our project studies BDH, a brain-inspired architecture positioned as an alternative design space to standard Transformers. The repository includes core model code, multi-scale memory variants, multiple training paths, distributed teacher-logits generation, and a live web app comparing BDH against DistilGPT2.
Technically, the core model uses linear attention-style computation and multi-scale decay settings in the implementation. Operationally, we built scripts for quick runs, long resumes, distillation, distributed data generation, benchmark reporting, and demo reliability. For judging readiness, we focused on reproducibility, limitations, and transparent claims.
We are not claiming this beats frontier LLMs. We are showing a research engineering contribution: architecture experimentation, practical training/deployment workflows, and critical evaluation with measurable outputs and clear limitations."
Use this framework in every answer:
- What we built
- What we measured
- What the current evidence supports
- What is still hypothesis/future work
Never blur these two classes:
- Webapp benchmark artifact reports BDH vs DistilGPT2 with:
- BDH perplexity: 85.3688
- DistilGPT2 perplexity: 33.2828
- BDH logic_score: 20
- DistilGPT2 logic_score: 0
- Source: benchmarking/results/webapp_bdh_vs_distilgpt2.json
- benchmark_results.json records baseline vs multiscale runs with small perplexity delta and throughput values.
- webapp/server.py contains CUDA-runtime detection and automatic CPU fallback + retry.
- webapp/server.py clips sampling logits to tokenizer vocab range to avoid out-of-range tokens.
- implementation/multiscale_bdh.py has RoPE code present but currently disabled in forward pass with TODO comments.
Many docs in README/presentation mention values like "13x", "24x", "300x", "95% stability", etc. Treat these as provisional unless reproduced live with scripts and logs.
Judge-safe phrasing: "Some internal planning docs include stronger target metrics; in this presentation we only stand behind numbers that are reproducible from current benchmark artifacts."
BDH in this repository is a research-oriented language-model stack with:
- Linear-attention style computation for sequence efficiency experimentation
- Multi-scale synaptic-state concept in code
- Multiple training/distillation workflows
- Demo and deployment infrastructure
- Multi-scale configuration in implementation/multiscale_bdh.py:
- decay_rates default: [0.95, 0.99, 0.995]
- scale_weights default: [0.2, 0.3, 0.5]
- hebbian_lr default: 0.001
- Multiplicative gating is used at layer level in MultiScaleBDHLayer.
- RoPE components exist, but application is commented out in forward pass.
- webapp server includes production-minded behaviors:
- model/tokenizer compatibility checks
- fallback tokenization path for small vocab
- runtime warnings surface
- CPU failover for CUDA runtime failures
Top-level purpose map for the current workspace:
- implementation/
- Purpose: Core multi-scale BDH implementation, tokenizer work, stability configs, training helpers.
- Key files:
- implementation/multiscale_bdh.py
- implementation/bbpe_tokenizer.py
- implementation/stable_config.py
- implementation/train_multiscale.py
- webapp/
- Purpose: Live backend + frontend demo comparing BDH and DistilGPT2.
- Key files:
- webapp/server.py
- webapp/static/index.html
- webapp/static/app.js
- webapp/static/styles.css
- webapp/run_professor_demo.ps1
- benchmarking/
- Purpose: Benchmark framework and result reporting.
- Key files:
- benchmarking/benchmark_runner.py
- benchmarking/statistical_analysis.py
- benchmarking/results/benchmark_results.json
- benchmarking/results/comparison_report.md
- benchmarking/results/webapp_bdh_vs_distilgpt2.json
- testing/
- Purpose: Integration and verification strategy notes.
- Key files:
- testing/integration_test.py
- testing/final_verification.md
- demo/
- Purpose: Stage-ready demo script, rehearsal notes, fallback plans.
- Key files:
- demo/demo_script.md
- demo/backup_plan.md
- presentation/
- Purpose: Poster/slides/talking-point preparation assets.
- Key files:
- presentation/slides.md
- presentation/poster_content.md
- presentation/talking_points.md
- presentation/judge_qa_prep.md
- research/ and research_reports/
- Purpose: Strategy, distillation best practices, roadmap and analysis notes.
- docs/
- Purpose: User and implementation docs.
- data/
- Purpose: Training/demo data assets (includes tinystories.txt).
- checkpoints/
- Purpose: Trained model checkpoints across multiple experiments and profiles.
- visualization/
- Purpose: Figures for poster/demo (retention curves, state matrix, training plots).
- Root-level train_.py and run_.{bat,ps1,py}
- Purpose: Many experiment entrypoints (quick, resume, distillation, hardware-specific runs).
Practical explanation to judges: "The repo is organized by lifecycle: implementation, training scripts, evaluation, demo surface, and communication assets."
- Environment setup scripts (setup_*.bat/.ps1/.sh) and guides.
- Pick a training mode (quick, safe, resume, distillation, hardware-specific).
- Run train_*.py script with checkpointing enabled.
- Monitor and resume using monitor/check/resume scripts.
- Evaluate with benchmark scripts and/or webapp quick benchmark endpoint.
- Teacher logits/data generation scripts:
- generate_logits.py, generate_logits_node*.py, distribute_logits_core.py
- Local or multi-node execution using run_logits_node*.ps1 and related scripts.
- Distillation training via train_distillation*.py and train_pure_distillation*.py variants.
- Save checkpoints for inference and webapp loading.
- Start web server (webapp/server.py or run_professor_demo.ps1 profile).
- Load BDH checkpoint and baseline model.
- Send prompt to /api/generate.
- Server returns side-by-side outputs + runtime metrics.
- If CUDA runtime fails, server switches to CPU and retries automatically.
From Regeneron ISEF Grand Award criteria, scoring is weighted:
- Research Question: 10
- Design and Methodology: 15
- Execution/Data Analysis: 20
- Creativity and Potential Impact: 20
- Presentation: 35
- Poster: 10
- Interview: 25
Key implication: Interview quality is heavily weighted. Your answer quality, ownership, limitation awareness, and clarity can decide outcomes.
- Authenticity: Did you really do this work?
- Independence: Can each teammate explain core decisions?
- Rigor: Do you separate evidence from assumptions?
- Integrity: Are you honest about limitations and failure modes?
- Impact thinking: Do you understand where this matters and where it does not?
Assign fixed primary roles:
- Teammate A: Problem + Architecture Lead
- Owns narrative, motivation, model design, literature context.
- Teammate B: Training + Systems Lead
- Owns scripts, data flow, checkpoints, reproducibility, hardware constraints.
- Teammate C: Evaluation + Ethics + Demo Lead
- Owns benchmarks, claim discipline, demo reliability, integrity/ethics answers.
- The first person gives a concise answer in <=20 seconds.
- Then says: "[Name] can add implementation detail" if needed.
- Avoid interrupting each other.
- Every answer ends with one concrete artifact reference (script, JSON, screenshot, run log).
If one person does not know, another person can add. Never bluff.
Judges usually ask questions for one of these reasons:
- Verify authenticity
- "How did you pick these decay rates?"
- They test if you did real design work.
- Stress test rigor
- "What controls/baselines did you run?"
- They test scientific method quality.
- Probe limitations
- "Where does this fail?"
- They test maturity and honesty.
- Evaluate relevance
- "Why should anyone care?"
- They test impact and practical understanding.
- Validate team contribution
- "Who implemented what?"
- They test whether all members contributed and understand the work.
Best response pattern:
- Context -> Exact action -> Evidence -> Limitation -> Next step
Use the format:
- Short answer (1-2 lines)
- Evidence anchor (file/artifact)
- Limitation line
- What exact problem are you solving?
- Why BDH instead of a standard Transformer experiment?
- What is novel in your current repo version?
- What did you personally contribute vs reuse?
- Why is this relevant for science, not just coding?
- Explain the multi-scale memory idea in simple terms.
- Why those decay rates?
- How does linear attention differ from softmax attention?
- Is your positional encoding active right now?
- What does multiplicative gating do here?
- How do state matrices update during inference/training?
- Is the model interpretable in a practical way?
- What tradeoffs did this architecture introduce?
- Where can this architecture underperform?
- What would you change first if you had one more month?
- How do I reproduce one of your runs?
- Which script is your most stable training path?
- How do you resume interrupted runs?
- What hardware did you target and why?
- What are your most common training failures?
- How do you detect overfitting/instability?
- How do you keep experiment tracking consistent?
- Which claims are from measured runs vs target goals?
- Why use distillation in this project?
- How do teacher logits get generated?
- Why distributed logits across nodes?
- How do you ensure shard consistency and resumability?
- What is your data quality control approach?
- What teacher-model bias risks exist?
- How do you validate distilled student behavior?
- Which benchmark numbers are verified right now?
- Did BDH beat DistilGPT2 on perplexity in your artifact?
- Why does logic_score differ from perplexity results?
- How do you avoid cherry-picking metrics?
- What benchmark is still missing for publication-level confidence?
- What happens if CUDA fails during live demo?
- How do you avoid tokenizer/model vocab mismatch errors?
- What if internet/model download fails at venue?
- How do you keep demo responsive under constraints?
- What is your fallback if the main demo breaks?
Question: "Did your model beat DistilGPT2?"
- Short answer: "On the current artifact, BDH did not beat DistilGPT2 on perplexity, but scored higher on our logic_score metric."
- Evidence: benchmarking/results/webapp_bdh_vs_distilgpt2.json
- Limitation: "This is a small sample benchmark and we treat it as directional, not final proof."
- Next step: "We are expanding evaluations and standardizing metric suites."
- Activate environment.
- Start server:
- python webapp/server.py --host 0.0.0.0 --port 8000 --checkpoint <chosen_checkpoint> --data data/tinystories.txt
- Open localhost:8000.
- Run 2 prepared prompts and one judge prompt.
- Hit /api/health.
- Generate once with short prompt.
- Confirm both BDH and Distil outputs render.
- Keep a screenshot backup of last successful run.
- Let auto CPU fallback happen (server already supports this).
- If needed, restart with a smaller/stable profile via webapp/run_professor_demo.ps1.
- Continue with pre-captured benchmark artifacts and explain transparently.
You can safely say:
- "We built an end-to-end research stack: model implementation, training scripts, distillation/logits pipelines, benchmarking, and web demo."
- "Our official benchmark artifacts currently show mixed outcomes: stronger logic_score for BDH in one artifact, but weaker perplexity than DistilGPT2."
- "We engineered robustness features, including CUDA-to-CPU failover and tokenizer-vocab safeguards."
- "We are actively separating verified measurements from aspirational targets in docs."
Do not claim without fresh proof:
- "13x/24x/300x improvements" as settled results.
- "State-of-the-art performance".
- "Scales to 1B+ demonstrated" unless you show run artifacts.
- "Always better than baselines".
Use these high-integrity responses:
- "Great point. We have not fully validated that yet. Here is what we have measured so far..."
- "That result appears in planning docs, but we are only claiming what is reproducible from current artifact files."
- "This is an active limitation in our implementation; here is the exact file where it appears and our planned fix."
- "We designed this as a research prototype, not as a production claim of superiority over frontier models."
These responses increase trust.
- Round 1 (45 min): Technical grilling
- One teammate plays aggressive judge.
- Use Section 10 only.
- Round 2 (30 min): Non-technical judge
- Explain architecture in simple language.
- Ban jargon unless defined in one sentence.
- Round 3 (20 min): Failure simulation
- Assume demo GPU failure and internet failure.
- Practice fallback narrative.
- Final 10 min: Claims lock
- Team agrees on exact safe claims and banned claims.
Primary sources:
- ISEF Grand Award criteria (official scoring + interview emphasis):
- ISEF judge page (judge qualifications, process, interview windows):
- ISEF international rules and AI guidance links:
- Society for Science scientific integrity policy:
- ISEF FAQ (judge profile and process context):
Secondary context source: 6. Reinventing Science Fairs (Issues in Science and Technology):
Your team does not need to claim perfection. Your team needs to demonstrate:
- Real ownership of work.
- Real understanding of tradeoffs.
- Real integrity in reporting evidence.
That combination is exactly what strong judges reward.