Senior Software Engineer: Ruby (LLM Evaluation & Repository Validation)

What you'll do

Ruby has a smaller pool of public repositories suited to benchmarking than Python or JavaScript, so the engineer who knows Rails-era conventions and gem layouts is useful here. The project builds checkable software engineering tasks from public repository histories, with a human validating each.

You would:

• Triage GitHub issues in popular open-source Ruby libraries
• Get each repository building and testing, including Docker and environment setup
• Assess unit test coverage and quality
• Modify and run the code locally to see whether an LLM can fix a bug
• Advise researchers on repositories and issues that are hard for models

Leading a team of junior engineers is described as a possibility.

Who fits

• 3 or more years of overall experience, with strong Ruby
• Git, Docker and basic pipeline setup
• Comfort with large, unfamiliar codebases and running them locally
• Open-source experience is a plus
• Nice to have: LLM research or evaluation experience, or building developer tools or automation agents

What it pays

The ad does not publish a rate. Turing sets rates per project; see Turing.

Commitment and location

• 20, 30 or 40 hours a week, minimum 4 hours a day, with 4 hours of overlap with PST
• Contractor assignment (no medical or paid leave)
• Locations named: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico
• No contract length is given

Worth knowing

Good:

• A niche language, which can mean less competition than for Python
• Real repository work you can describe on a CV
• Remote with a choice of weekly hours

Less good:

• No rate published
• Ruby projects with solid test suites are fewer, so the project may lean on a small set of repositories (the ad does not say)
• Pacific time overlap and country limits
• No benefits or paid leave, and no stated term

How hiring works

Two interviews, about 75 minutes in total: 60 minutes technical and 30 minutes technical and cultural.

About this listing

Read from Turing's job board on 18 October 2026. The hours, locations and interview structure are Turing's own text. See Turing.