Sign in

Software Engineer, Python - Codebase Q&A

Weekday AI · Remote

The role

This contract role involves creating difficult codebase-exploration tasks for AI agents, focusing on production Python repositories. The work is task-based, remote, and paid per approved task. Engineers must deeply understand codebases and write precise questions and rubrics.

Location
Remote
Work mode
Remote
Employment
Contractor
Level
Senior
Experience
3+ yrs

What we know that the posting doesn’t say

  • Seen todaystill listed on the employer’s careers page
  • Posted 3 days agothe first time we saw it
Apply with ApivaApply on the company site

About Weekday AI

Weekday AI is hiring for a client for a role building datasets to train AI agents to reason about codebases.

What you would do

  • Select tasks from a pre-validated pool of Python repositories.
  • Explore repositories to understand subsystems and relevant history.
  • Rewrite task questions to challenge frontier coding agents.
  • Adjust rubrics and foils to reward correct reasoning.
  • Run the validation loop in Studio until all stages pass.
  • Document work in task README and submit for expert review.

Must have

  • 3+ years of professional software engineering experience.
  • Substantial production Python experience.
  • Ability to navigate large, unfamiliar Python codebases.
  • Explain how codebases behave at runtime, not just source.
  • Fluency in Python internals like import machinery and metaclasses.
  • Experience with async and event loops.
  • Knowledge of packaging and C-extension boundaries.
  • Precise written English for questions and rubrics.
  • Comfortable with git at commit level.
  • Comfortable with containerized environments.
  • Comfortable with command-line tooling.
  • Pass a short calibration quiz before task access.

Nice to have

  • Experience with web frameworks.
  • Experience with developer tooling.
  • Experience with type checkers.
  • Experience with scientific or numeric libraries.

What you get

  • Fully remote and asynchronous work.
  • Expert team leads across US, India, and Nigeria time zones.
  • Payment per approved task.
  • Reasonable accommodations upon request.

Experience and education

  • 3+ years of relevant experience

Key skills

  • python
  • django
  • twisted
  • scrapy
  • sympy
  • mypy
  • pip
  • coverage.py
  • mitmproxy
  • certbot
  • tornado
  • werkzeug
  • sqlfluff
  • meson
  • git
  • containers
  • command-line
  • async
  • event loops
  • import machinery
  • descriptors
  • metaclasses
  • packaging
  • c-extension

This role is for one of our clients Compensation: $130 per task We are seeking experienced Python engineers to build the dataset that teaches AI agents to reason about real codebases. Codebase Q&A is a reinforcement-learning environment for training AI agents to explore and reason about software re…

Extracted from the employer’s posting. Read it in full on Weekday AI’s careers page

Apply without filling the form

Approve this role and the application is completed for you, including a résumé tailored to it. You get a confirmation when it lands, and a credit is only spent when a submission is confirmed.

Claim 25 free applications

Other roles at Weekday AI

All 8 roles at Weekday AI →