AIRLINE BENCHMARK · TEST SUMMARY
What the offline tests verify
The adapter tests use a fake model and a real MCP server to check execution behavior without paid model calls.
| Check | Behavior verified |
|---|---|
| Native request parity | The adapter constructs model requests in the same way as the native agent. |
| MCP result delivery | The model receives the MCP server’s tool result. |
| Failure handling | MCP failures stop execution rather than triggering a local fallback. |
| Concurrent session isolation | Three conversations maintain separate sessions and tool state. |
| Continuation diagnostics | Saved histories and restored tool state are checked when selected conversations are continued after removing only their final STOP token. |
Recorded test results
The initial adapter implementation passed 72 offline tests. After adding eight continuation-diagnostic checks, the expanded suite passed 80 tests. Lint and format checks also passed.
Independent live checks
Single and concurrent smoke tests exercised live model execution. The full MCP baseline then completed 200 conversations with zero MCP infrastructure errors and zero tool-result mismatches. Those live measurements are separate from the offline test suite.
Offline tests verify implementation behavior. They do not predict a model’s benchmark score or prove that two systems have equivalent success rates. This page summarizes previously recorded checks; preparing it did not rerun the tests.