Operate and stress-test LLM-based AI tools, reproduce and document edge cases, file and track tickets, validate fixes, provide prioritized product feedback, triage failures with engineers, design lightweight tests, and support the Head of Engineering with task and calendar management to scale experimentation and improve production reliability.
About the Client
The hiring manager is the Head of Engineering, who partners closely with product team and iterate quickly on emerging LLM workflows. The culture values autonomy, candor, and high standards in a startup-paced environment.
The hiring manager is the Head of Engineering, who partners closely with product team and iterate quickly on emerging LLM workflows. The culture values autonomy, candor, and high standards in a startup-paced environment.
Why does this role exist?
This role exists to serve as a highly autonomous right hand to the Head of Engineering, operating and stress-testing experimental AI tools while closing the feedback loop between users and the engineering/product teams. By proactively finding breakages, documenting issues, and prioritizing work across a 30+ person team, this role accelerates engineering velocity and product quality. It ensures the organization can scale experimentation, reduce failures in production, and maintain tight operational rhythms without relying on rigid SOPs.
The Impact you’ll make
AI Tool Operations & Testing
- Operate, stress-test, and quality-check cutting-edge AI tools and LLM-based systems that may frequently break
- Reproduce edge cases, capture logs/screens, and write clear repro steps; file/track tickets in Shortcut (Clubhouse)
- Validate fixes and run smoke/regression tests before and after deployments
Product Feedback & Collaboration
- Provide structured, actionable feedback to the Head of Engineering and product/engineering teams
- Collect and synthesize feedback from other assistants/users; turn insights into prioritized improvements
- Critique internal software and propose data-backed enhancements to workflows and UX
Technical Troubleshooting & QA
- Triage broken workflows, diagnose likely failure points, and collaborate with engineers on root causes
- Consolidate deployment updates and confirm changes behave as expected in production
- Design lightweight test plans for ambiguous features; track outcomes and follow through to resolution
Operational Support to the Head of Engineering
- Manage and organize the Head of Engineering’s to-do list; drive proactive follow-through across a 30+ person team
- Maintain and optimize work queues; ensure the right priorities are surfaced at the right time
- Handle calendar management via AI tools; coordinate with stakeholders as needed
Systems & Process Optimization
- Build simple, scalable processes in ambiguous environments; streamline task flows and handoffs
- Propose and implement improvements to reduce repeated failure modes and increase throughput
- Leverage AI productivity tools to automate routine steps and create operational leverage
Skills, Knowledge and Expertise
Required:
- Demonstrated technical mindset with strong AI tool fluency (e.g., ChatGPT, LLM workflows) or a programming background
- Proven experience in product operations, QA/Testing, technical support, or similar role working closely with engineers
- Exceptional written communication for clear, structured bug reports and product feedback
- Comfortable operating independently in ambiguous, fast-paced environments with minimal SOPs
- Availability to work 40 hours/week, Monday–Friday, 9:00 am–5:00 pm Eastern Time (may shift slightly earlier after 3–4 weeks)
WFH Set-Up:
- Computer with at least 8GB RAM, an Intel i5 core processor/AMD Ryzen 5 Processor and up.
- Internet speed of at least 40MBPS
- Headset with an extended mic that has noise cancellation and a webcam
- Back-up computer and internet connection
- Quiet, dedicated workspace at home
Your Superpowers:
- Technical: AI tool proficiency; systems thinking; test design and execution; QA methodologies; debugging mindset; clear documentation; comfort with Shortcut (Clubhouse), GSuite, Slack, and experimental internal tools (e.g., Magic AI); familiarity with messaging tools (e.g., Messenger). Bonus: basic scripting/automation, product analytics literacy.
- Operational: Task prioritization; work queue management; lightweight process design; deployment update consolidation; calendar management via AI.
- Behavioral: Proactive ownership; “figure it out” mentality; high attention to detail; resilience when tools break; curiosity and growth mindset; comfort giving candid, constructive feedback; ability to work independently and collaboratively with engineers.
You should apply if...
- You love being an early power user of new AI tools, enjoy breaking things to make them better, and can translate ambiguous symptoms into actionable engineering feedback.
- You thrive in uncertainty, don’t need handholding, and naturally create order and leverage through systems and process improvements.
- You communicate crisply in writing, challenge assumptions respectfully, and partner well with engineers and product managers.
- You want to grow at the intersection of AI operations, product testing, and systems thinking. Bonus: You’re in a LATAM timezone for strong overlap with EST, and/or you have experience in programming, product, or data analytics.
What to expect...
Work Setup:
- Remote position
- Must have a reliable internet connection and a quiet workspace
- Required to provide own computer with Intel Core i5 or something similar or higher operating system
Working Hours:
- 40 hours per week
- M-F 9am to 12 noon, 1 pm to 4 pm US Timezone
Compensation:
- $7 per hour
- No benefits package included
Benefits
About
Magic has connected top remote talent with fast-growing businesses for over 10 years.Founded in San Francisco in 2015, we now have thousands of remote workers around the world. Magic is backed by Sequoia Capital and Y Combinator.
Similar Jobs
Fintech • Mobile • Payments • Software • Financial Services
Lead Canada regulatory and financial crime compliance programs: engage regulators, horizon-scan Canadian legislation, maintain 2LoD FinCrime policies (KYC/EDD/transaction monitoring), advise product teams for compliant-by-design features, oversee monitoring/testing and internal audits, manage remediation and board reporting, and collaborate cross-functionally to embed scalable compliance controls.
Top Skills:
ConfluenceGoogle SuiteJIRAOfficeSlackTeams
Artificial Intelligence • Productivity • Software • Automation
Lead Zapiers security strategy for an AI-native SaaS platform. Own App and Infrastructure Security, Detection & Response, GRC, risk management, incident response, bug bounty triage, and enterprise security enablement. Partner with Product, Engineering, Legal, and GTM to embed secure-by-default design, shape AI-specific controls, support enterprise deals, and build a high-performing security engineering organization.
Top Skills:
AIBug Bounty PlatformsCloudDetection & ResponseForensicsIdentity And Access Management (Iam)Incident ResponseSecure SdlcThreat Modeling
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Lead and coach a team of ~8 account executives selling full-cycle to SMBs: hire, onboard, train, review pipeline, support deals, create training content, and collaborate cross-functionally to meet quarterly targets.
Top Skills:
Salesforce (Sfdc)
What you need to know about the Vancouver Tech Scene
Raincouver, Vancity, The Big Smoke — Vancouver is known by many names, and in recent years, it has gained a reputation as a growing hub for both tech and sustainability. Renowned for its natural beauty, the city has become a magnet for professionals eager to create environmental solutions, and with an emphasis on clean technology, renewable energy and environmental innovation, it's attracted companies across various industries, all working toward a shared goal: advancing clean technology.
