On this page

A coding agent can turn a written request into files, run the code, repair an error, and report that every test passed. In research, that result can still be wrong.

The program may have selected the wrong variable, combined raw values with adjusted-data quality flags, changed a unit, filled missing values with zero, or implemented a formula that differs subtly from the intended method. None of those mistakes has to produce an exception. A plausible graph and a successful command can hide them.

Codex and Claude Code are useful because they can work across a repository rather than returning one isolated code fragment. They can inspect existing code, edit several files, run commands, add tests, and summarize a diff. The benefit is faster implementation and review—not automatic scientific judgment.

This guide is for researchers who want that help but do not maintain software professionally. It shows how to give a coding agent a controlled workspace, define the scientific method before implementation, and require evidence stronger than “the code runs.”

What a Coding Agent Changes

A normal chat assistant usually gives you text or a code block that you copy elsewhere. A coding agent can operate in the project itself.

CapabilityResearch benefitRequired check
Read the repositoryAdapts to existing loaders, tests, and conventionsConfirm it read the correct method and data documentation
Edit several filesKeeps code, tests, and documentation togetherReview the complete diff
Run commandsFinds syntax, import, and execution failuresUse the declared environment and inspect command output
Write testsMakes assumptions executable and repeatableDetermine expected scientific behavior independently
Revise after failureSpeeds up debuggingEnsure it did not weaken the test or change the method

The agent is well suited to implementation, refactoring, test scaffolding, documentation, and routine checks. The researcher should retain decisions such as:

  • Which scientific question is being answered
  • Which variables and units represent that question
  • Which quality-control policy is acceptable
  • Which equation, interpolation method, or statistical model is appropriate
  • What expected result is independent enough to validate the implementation
  • Whether the evidence is sufficient to use the output

Build the Foundation Before Starting the Agent

An agent is easier to control when the project already has three layers:

  • Git and GitHub: Record what changed and support review
  • Docker: Record where and how the analysis runs
  • Method document: Record what the analysis is supposed to calculate

If those layers are new to you, begin with:

The agent should work on a Git branch so every change is visible. Analysis and tests should run through the documented Docker command so the agent does not silently depend on packages installed only on the host.

Give the Agent a Controlled Project

Keep code, instructions, tests, and small non-sensitive fixtures in one repository. Keep irreplaceable or restricted raw data outside the agent’s writable workspace.

research-project/
├── AGENTS.md
├── CLAUDE.md
├── README.md
├── Dockerfile
├── compose.yaml
├── requirements.txt
├── method/
│   └── analysis-contract.yaml
├── src/
├── tests/
│   └── fixtures/
└── output/

research-data/              # Outside the writable repository

Mount the external data read-only when the Docker analysis runs. Store credentials outside the repository, and do not paste secrets into prompts or instruction files.

An instruction file is context for the model, not an access-control boundary. Use the agent’s sandbox and approval settings, operating-system permissions, and read-only mounts to enforce the boundary.

Install the Agent Separately from the Analysis

Follow the current official installation and authentication instructions for the agent you choose. Start it from the repository root:

cd research-project

# Start one tool in this working tree
codex

# Or use Claude Code instead
claude

Do not install the interactive agent inside the scientific Docker image merely to make the command available. Coding agents change frequently; the analysis image is intended to preserve a reviewed runtime.

Do not let two agents edit the same working tree simultaneously. Use separate Git branches or worktrees if you want independent implementations or a second-agent review.

Put Durable Rules in the Repository

Codex reads repository guidance from AGENTS.md. Claude Code reads CLAUDE.md; its official documentation supports importing an existing AGENTS.md.

Create a concise AGENTS.md:

# Research analysis instructions

## Scientific contract

- Treat raw input data as immutable.
- Do not choose raw or adjusted variables silently.
- Record variable names, units, quality filters, and missing-value rules.
- Keep pressure in dbar unless an approved method defines a conversion.
- Ask before changing a threshold, equation, interpolation method,
  coordinate convention, or statistical model.

## Working rules

- Run analysis and tests through Docker Compose.
- Write generated files only under `output/`.
- Add a small known-answer test for each derived calculation.
- Never change an expected value merely to make a test pass.
- Report assumptions, failed checks, and unresolved scientific choices.

## Done when

- The requested code and tests are implemented.
- The documented checks pass.
- The diff has been reviewed for units, filters, equations, and tolerances.
- Output metadata identifies the data snapshot, parameters, and Git commit.

Create CLAUDE.md beside it when the same repository supports Claude Code:

@AGENTS.md

Keep these rules short and specific. A textbook-sized instruction file competes with the task context and is harder to keep accurate. Put the detailed scientific definition in a separate method document.

Write the Analysis Contract Before the Code

For a consequential calculation, write down the approved choices before asking the agent to implement them:

question: Compare upper-ocean temperature across selected profiles

inputs:
  snapshot: "provider, retrieval date, and checksum go here"
  pressure:
    variable: PRES_ADJUSTED
    unit: dbar
  temperature:
    variable: TEMP_ADJUSTED
    unit: degree_Celsius

quality:
  accepted_flags: ["1", "2"]
  missing_values: reject

method:
  pressure_interval_dbar: [0, 500]
  interpolation: none
  aggregation: none

validation:
  - dimensions and units match this contract
  - pressure is strictly increasing after filtering
  - a synthetic profile has an independently calculated answer
  - representative real profiles are inspected visually

This contract separates scientific decisions from implementation decisions. The agent may identify ambiguity, but it should not silently decide between TEMP, TEMP_ADJUSTED, and several QC arrays.

If the method is still uncertain, ask for an inspection and plan first. Resolve the scientific choices before authorizing implementation.

Give a Task with an Acceptance Test

You do not need elaborate prompt engineering. State the goal, governing context, constraints, validation, and completion condition.

Goal:
Implement method/analysis-contract.yaml.

Context:
Read AGENTS.md, the method contract, and the existing loaders under src/.

Constraints:
Do not modify raw data, the method contract, or existing expected results.
Run Python only through Docker Compose.
Stop and report if a required variable or unit is unavailable.

Validation:
Add synthetic tests whose expected values were determined independently.
Run focused tests and inspect one representative output for impossible values.

Done when:
Show changed files, commands run, test results, assumptions, and every
scientific decision that still needs human approval.

This structure makes omissions visible. If the agent cannot point to an approved variable, expected value, or completion check, the task is not ready to accept.

Separate Structural Checks from Scientific Checks

A first test can check whether the input has the expected shape:

from math import isfinite


def validate_profile(
    pressure_dbar: list[float],
    temperature_c: list[float],
) -> None:
    if len(pressure_dbar) != len(temperature_c):
        raise ValueError("pressure and temperature lengths differ")
    if len(pressure_dbar) < 2:
        raise ValueError("profile needs at least two levels")
    if not all(isfinite(value) for value in pressure_dbar + temperature_c):
        raise ValueError("profile contains a non-finite value")
    if any(
        upper <= lower
        for lower, upper in zip(pressure_dbar, pressure_dbar[1:])
    ):
        raise ValueError("pressure must be strictly increasing")

Its tests might include:

import unittest

from src.profile_checks import validate_profile


class ValidateProfileTest(unittest.TestCase):
    def test_accepts_increasing_pressure(self) -> None:
        validate_profile([0.0, 10.0, 20.0], [24.0, 22.0, 19.0])

    def test_rejects_mismatched_lengths(self) -> None:
        with self.assertRaisesRegex(ValueError, "lengths differ"):
            validate_profile([0.0, 10.0], [24.0])

    def test_rejects_reversed_pressure(self) -> None:
        with self.assertRaisesRegex(ValueError, "strictly increasing"):
            validate_profile([0.0, 20.0, 10.0], [24.0, 19.0, 22.0])

These tests catch malformed arrays. They do not prove that a mixed-layer calculation, regression, interpolation, or water-mass classification is scientifically correct. Structural validation and scientific validation answer different questions.

Use an Independent Known Answer

Every derived calculation should have at least one case whose answer was determined independently of the generated implementation.

Suitable sources include:

  • A hand calculation small enough to audit line by line
  • A synthetic profile designed to have an obvious result
  • A reviewed reference implementation
  • A published worked example with matching definitions
  • A validated pipeline output with known provenance

An agent may write the test code. Do not ask the same agent to invent the algorithm and then accept its own output as the independent expected value.

Set numerical tolerances from the method and precision. If a test fails, investigate units, constants, preprocessing, definitions, and floating-point behavior. Do not broaden a tolerance only to obtain a green test.

Validate in Layers

Review agent-generated analysis in this order:

  1. Execution: Does the documented command complete in the declared environment?
  2. Data contract: Do dimensions, coordinates, units, types, and missing values match the method?
  3. Known answer: Does a small input produce an independently determined result?
  4. Reference comparison: Does representative data agree with a trusted implementation or worked example?
  5. Sensitivity: Do reasonable parameter changes produce understandable behavior?
  6. Scientific inspection: Do intermediate values, profiles, maps, and distributions make physical sense?
  7. Reproduction: Can a clean checkout recreate the claimed output from the documented inputs and command?

Passing one layer does not imply that later layers pass. Software tests are necessary evidence, not a substitute for scientific review.

Review the Diff Before the Figure

Inspect the Git diff before accepting a polished final plot. Pay particular attention to:

  • Equations, constants, and numerical tolerances
  • Unit conversions
  • Raw, adjusted, and QC-variable selection
  • Filtering and sorting order
  • Missing-value handling
  • Interpolation across unsupported gaps
  • Changes to method documents and expected test values

Ask the agent to map each scientific change to the method requirement it implements. If it cannot do that, do not accept the change merely because the graph looks plausible.

Commit only after the tests, representative outputs, and diff have been reviewed. Small commits make it possible to identify when a scientific choice entered the code.

Keep Permissions Narrow

Start with the tools’ normal sandbox and approval controls:

  • Allow writes only inside the research repository
  • Keep raw or restricted data outside that writable area
  • Require approval for network access and commands outside the project
  • Deny access to credential files and secret directories
  • Review unfamiliar package installation, download, shell, and Docker commands
  • Do not expose the Docker socket to an untrusted container

Codex distinguishes the sandbox, which limits what commands can access, from the approval policy, which determines when the user must authorize an action. Claude Code provides permission rules and deny patterns. Repository instructions guide behavior; enforced permissions provide the technical boundary.

Watch for Errors That Still Produce Plausible Output

Ocean-data analysis has many mistakes that may not crash:

  • Pairing raw values with adjusted-data QC flags
  • Treating pressure in dbar as exact geometric depth in metres
  • Mixing practical salinity, Absolute Salinity, in-situ temperature, potential temperature, and Conservative Temperature
  • Replacing fill values with zero
  • Sorting one variable without applying the same order to related variables
  • Interpolating across a large unsupported gap
  • Treating a drifting float’s sequence as a fixed-location time series

Argo Data Quality Control: QC Flags, Adjusted Data, and Data Modes explains the variable and QC relationship. Argo NetCDF Format Explained for Beginners explains the file structure behind those fields.

Use OceanGraph as a Separate Visual Check

OceanGraph can provide a separate first-pass view of selected Argo profiles before or after custom code is written. Check the location, cycle, vertical structure, and whether a generated pattern resembles the profiles chosen for analysis.

This is not numerical validation of a custom algorithm. It is another way to notice a reversed axis, missing layer, implausible profile choice, or mismatch between the scientific question and the selected data.

A Practical Definition of Done

Agent-assisted analysis is ready for human acceptance only when:

  • The method was specified before implementation
  • Inputs, units, quality rules, and parameters are recorded
  • The agent worked in a reviewable Git diff
  • Commands ran in the declared environment
  • Known-answer and reference checks passed
  • Representative intermediate and final outputs were inspected
  • No expected value or method rule was changed only to make a test pass
  • Remaining uncertainty and required domain review are explicit

The agent can accelerate the work behind every item. It cannot take responsibility for the final scientific judgment.

If setting up this validation workflow for agent-generated analysis code inside your own team would be difficult, the inquiry link below can also be used to ask about setup and implementation support.

Contact

Implementation support and annotation services

Ask about setup, design, and implementation support, or annotation services.

Go to the contact form

Continue with:

Product and research references:

The linked benchmark covers one research field and task set. It is not a universal accuracy estimate for every analysis. Its relevance here is narrower: generated research code needs an explicit plan and independent validation.