This is an as-told-to essay based on a conversation with Frank Jones, a management consultant analyst for Mercor, which employs more than 30,000 human contractors to train AI models for Big Tech companies. This story has been edited for length and clarity.
I started AI training for Mercor last year because I wanted to work from home, and it’s now my primary job.
I previously worked at a Big Four consulting firm, so I work on training AI on consulting tasks. I love it because the focus is really on the work. There’s not a lot of calls or meetings. It’s a lot of working independently.
I’ve worked on several different training projects, and there are different approaches to each one.
I like to use the metaphor of making a cake. In one approach, you give the AI model the recipe, or instructions, on how to bake the cake, and you let AI bake it. Then you taste it, and measure it according to a grading scale you created. How sweet is it? Is it fluffy? The rubric for a consulting task might be: Is this professional? Are the numbers accurate? How is it formatted? Those types of things.
In another approach to training, I’ll write down my recipe and bake a cake myself cake. I’ll also have AI bake a cake, too. Then I’ll taste both cakes and determine which is better.
I see AI as a supplement to consultants, not a replacement. What we are doing with writing prompts, rubrics, and reviewing are things that consultants will still be needed for even with AI tools.
AI is best at straightforward tasks, but human review is still needed
When it comes to consulting tasks, AI works best with things that are more straightforward or black and white. The challenge is how does it deal with nuance. Like if I go to my cake analogy: a dash of sugar, a pinch of salt, take it out of the oven when it looks like it’s risen.
In consulting, you may have a strict calculation if you’re writing a proposal for a client, and AI could accurately price something at $267,000. But a human in the real world might say, “Well, actually, I want it to be under $250,000,” and know to play with it because that figure will sound better. It’s not a rule, but human judgment.
With many tasks, AI gets you close, but often some refinement or tweaks are needed. I’m not sure if that’s something that will ever be completely solved for. You can get closer, so there’s less tweaks, but even if you think about an automated car plant, you still need that quality inspector who comes in and makes sure that the automation is working as it should.
AI prompting will be a necessary skill in consulting
Prompting is going to be its own skill. You can’t just clap your hands and get the response you want. You have to be able to tell an AI tool what you want and provide the right input.
Working with AI has made me realize the importance of being specific with language. Some things you take for granted and think “everybody knows this,” but the meaning might be more obscure or ambiguous than you thought.
For example, “client-ready” is a common phrase in consulting. “This presentation needs to be client-ready.” Most consultants know what that means. But for AI, there’s a lot of nuance.
Even when you do your analysis, whether it’s AI or not, the job of a consultant isn’t just to hand the result over to clients. It’s also to help synthesize it for them, help them understand it, draw the right conclusions, and come up with the right recommendations.
Like with any technology, there’s a shift in the type of jobs and how humans are needed, but it’s not a complete replacement.
Have a tip? Contact this reporter via email at kvlamis@businessinsider.com or Signal at @kelseyv.21. Use a personal email address, a nonwork WiFi network, and a nonwork device; here’s our guide to sharing information securely.
Read the full article here
