First live test with CATIA

1. Preparation (5 min)

1. Get the latest version (unzip the new ZIP file from Martin or run git pull) and run scripts\setup_user.cmd again. At the end it must say: ALL TESTS PASSED — 86 tools ready.
2. Start CATIA normally (not “Run as administrator”). Use the English user interface if possible; otherwise test anyway and note the language.
3. Open a Command Prompt (cmd, not as administrator) in the project folder and install the test tool once:

   .venv\Scripts\python.exe -m pip install "pytest>=7.0"
   mkdir docs\live

2. Automated tests (15 min)

All output goes to the folder docs\live. Do not use CATIA while the tests are running.

1. Record the environment (read-only – CATIA version, service pack, Python):

   .venv\Scripts\python.exe scripts\live_env_report.py --catia --out docs\live\umgebung.json

   The file must contain "status": "attached". If not, CATIA is not running or not running with the same rights – see section 5.
2. Acceptance test – checks items 2 to 11 of the checklist automatically (creates its own parts MCP_Abnahme_* and closes them without saving) and writes the report docs\live\abnahme.md:

   .venv\Scripts\python.exe scripts\live_acceptance.py --out docs\live --record docs\live\abnahme.jsonl

   FAIL here means “finding found”, not “broken”. The acceptance test never starts CATIA itself – if no CATIA session is running, C1 reports “no running CATIA; not launched”. Please take a quick look at the report: if C1 does not show PASS, CATIA cannot be reached – see section 5. The line “Offen gebliebene Testdokumente” (test documents left open) lists MCP_Abnahme_* parts that could not be closed – please close them in CATIA without saving.
3. Functional tests against CATIA with recording of the CATIA calls (paths are anonymised):

   set CATIA_MCP_RECORD=docs\live\aufzeichnung.jsonl
   .venv\Scripts\python.exe -m pytest tests\test_workflows.py --catia-live -rs -v > docs\live\tests.txt 2>&1
   set CATIA_MCP_RECORD=

   Failures are not a problem here – they are exactly the findings we are looking for. With a German user interface, tests may fail because of different feature names – please send the results anyway.
4. Surface test from the upstream project (creates a part, saves a test part and an image in the project folder):

   .venv\Scripts\python.exe scripts\gsd_smoke_test.py > docs\live\gsd.txt 2>&1

3. Test with Copilot (20 min)

Open VS Code with the project folder, open .vscode\mcp.json and start the server catia-v5 there, then set Copilot Chat to Agent mode. Enter the prompts one after the other, confirm each tool call and enter the result in the checklist – and also check in CATIA.

1. “Create a new part: plate 100 × 60 × 5 mm on the XY plane, centred on the origin. Give the volume, centre of gravity and bounding box.”
2. “Sketch a circle with radius 10 in the centre of the XY plane and set a radius constraint of 12 mm.” → Check in CATIA: is it a radius constraint, and has the circle changed to R12?
3. “New part: rectangle 100 × 60 on XY, then a 10 mm pad, mirrored extent. Give the volume.”
4. “Show the journal and undo the last change.” → Check in CATIA: is the last feature gone?
5. “Save the active part.” → Expected: refused (original/unsaved). Then: “Save a working copy as Livetest_Platte.CATPart in the working folder.”
6. “Export the working copy as STEP to the working folder.”

4. Checklist

Rows 2–9 are already checked automatically by the acceptance test (see abnahme.md). Please still look in CATIA during the Copilot test (section 3) and fill them in – a person sees more than the measured values.

5. If something does not work

6. Send the results back

1. Send the folder docs\live as a ZIP to Martin (contains umgebung.json, abnahme.md/abnahme.json, abnahme.jsonl, tests.txt, gsd.txt, aufzeichnung.jsonl). The work-* subfolders with the test parts should not be included.
2. The completed checklist (a photo is fine) and Copilot’s answers to prompts 1, 3 and 4 (copied).
3. Do not send any CATPart, STEP or image files. The recording contains names and dimensions of the test parts, but no paths.

Clean-up afterwards: the test part and image from the surface test in the project folder (GSD_SmokeTest*.CATPart, gsd_smoke_test.jpg) and the files in the working folder can be deleted.