feat - tests + evals

.github/workflows/evals.yaml

@@ -0,0 +1,24 @@
+name: Run MCP Evaluations
+on:
+  pull_request:
+    types: [opened]
+jobs:
+  evaluate:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20'
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Run MCP Evaluations
+        uses: matthewlenhard/[email protected]
+        with:
+          evals_path: 'src/evals/evals.ts'
+          openai_api_key: ${{ secrets.OPENAI_API_KEY }}
+          model: 'gpt-4' 

README.md

@@ -62,6 +62,15 @@ npx -y @smithery/cli install iterm-mcp --client claude
 ```
 [![smithery badge](https://smithery.ai/badge/iterm-mcp)](https://smithery.ai/server/iterm-mcp)
 
+
+
+## Running evals
+
+The evals package loads an mcp client that then runs the index.ts file, so there is no need to rebuild between tests. You can load environment variables by prefixing the npx command. Full documentation can be found [here](https://www.mcpevals.io/docs).
+
+```bash
+OPENAI_API_KEY=your-key  npx mcp-eval src/evals/evals.ts src/index.ts
+```
 ## Development
 
 Install dependencies: