Bounds bounds bounds bounds. The important part for humans seems to be maintaining boundaries for AI. If your well-tested codebase has the tests built thru AI, its probably not going to work.
I think its somewhat telling that they can't share numbers for how they're using it internally. I want to know that Microsoft, the company famous for dog-fooding is using this day in and day out, with success. There's real stuff in there, and my brain has an insanely hard time separating the trillion dollars of hype from the usefulness.
In any case, I think this is the best use case for AI in programming—as a force multiplier for the developer. It’s for the best benefit of both AI and humanity for AI to avoid diminishing the creativity, agency and critical thinking skills of its human operators. AI should be task oriented, but high level decision-making and planning should always be a human task.
So I think our use of AI for programming should remain heavily human-driven for the long term. Ultimately, its use should involve enriching humans’ capabilities over churning out features for profit, though there are obvious limits to that.
[0] https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...
So far, the agent has been used by about 400 GitHub employees in more than 300 our our repositories, and we've merged almost 1,000 pull requests contributed by Copilot.
In the repo where we're building the agent, the agent itself is actually the #5 contributor - so we really are using Copilot coding agent to build Copilot coding agent ;)
(Source: I'm the product lead at GitHub for Copilot coding agent.)
The actual quote by Satya says, "written by software".
Most developers don't love writing tests, or updating documentation, or working on tricky dependency updates - and I really think we're heading to a world where AI can take the load of that and free me up to work on the most interesting and complex problems.
where are we wrt the agent surveying open issues (say, via JIRA) and evaluating which ones it would be most effective at handling, and taking them on, ideally with some check-in for conirmation?
Or, contrariwise, from having product management agents which do track and assign work?
In this context, assuming that humans will still be able to do high level planning anywhere near as well as an AI, say 3-5 years out, is almost ludicrous.
The entire website was created by Claude Sonnet through Windsurf Cascade, but with the “Fair Witness” prompt embedded in the global rules.
If you regularly guide the LLM to “consult a user experience designer”, “adopt the multiple perspectives of a marketing agenc”, etc., it will make rather decent suggestions.
I’ve been having pretty good success with this approach, granted mostly at the scale of starting the process with “build me a small educational website to convey this concept”.
Similar to google. MS now requires devs to use ai
I'm curious to know how many Copilot PRs were not merged and/or required human take-overs.
Really cool, thanks for sharing! Would you perhaps consider implementing something like these stats that aider keeps on "aider writing itself"? - https://aider.chat/HISTORY.html
What is the job for the developer now? Writing tickets and reviewing low quality PRs? Isn't that the most boring and mundane job in the world?
every bullet hole in that plane is the 1k PRs contributed by copilot. The missing dots, and whole missing planes, are unaccounted for. Ie, "ai ruined my morning"
Everyone who has used AI coding tools interactively or as agents knows they're unpredictably hit or miss. The old, non-agent Copilot has a dashboard that shows org-wide rejection rates for for paying customers. I'm curious to learn what the equivalent rejection-rate for the agent is for the people who make the thing.
It is difficult to get a man to understand something when his salary depends upon his not understanding it. - Upton Sinclair
How does this align with Microsoft's AI safety principals? What controls are in place to prevent Copilot from deciding that it could be more effective with less limitations?
So they won’t like working on their job ?
Who would've thought (except you) that this would be one of the things that AI would be especially suited for. I don't know what this progression means in the long run. Will good engineers just become 1000x more productive as they manage X number of agents building increasingly complex code (with other agents constantly testing, debugging, refactoring and documenting them) or will we just move to a world where we just have way fewer engineers because there is only a need for so much code.
I use all of these tools, but you also know what "they're doing"...
I know our careers are changing dramatically, or going away (I'm working on a replacement for myself), but I just like listening to all the "what we're doing is really helping you..."
This was true up around 15 years ago. Hasn't been the case since.
Well, that's back rationalization. I saw the advances like conducting meta sentiment analysis on medical papers in the 00's. Deep learning was clearly just the beginning. [0]
> Who would've thought (except you)
You're othering me, which is rude, and you're speaking as though you speak for an entire group of people. Seems kind of arrogant.
0. (2014) https://www.ted.com/talks/jeremy_howard_the_wonderful_and_te...
My view is in between yours: A bit of column A and B in the sense both outcomes to an extent will play out. There will be less engineers but not by the factor of productivity (Jevon's paradox will play out but eventually tap out), there will be even more software especially of the low end, and the ones that are there will be expected to be smarter and work harder for the same or less pay grateful they got a job at all. There will be more "precision and rigor", more keeping up required by workers, but less reward for the workers that perform it. In a capitalist economy it won't be seen as a profession to aspire to anymore by most people.
Given most people don't live to work, and use their career to also finance and pursue other life meanings it won't be viable for most people long term especially when other careers give "more bang for buck" w.r.t effort put into them. The uncertainty in the SWE career that most I know are feeling right now means to newcomers I recommend on the balance of risk/reward its better to go another career path especially for juniors who have a longer runway. To be transparent I want to be wrong, but the risk of this is getting higher now everyday.
i.e. AI is a dream for the capital class, and IMO potentially disastrous for social mobility long term.
I'd like a breakdown of this phrase, how much human work vs Copilot and in what form, autocomplete vs agent. It's not specified seems more like a marketing trickery than real data
Those orgs that value high-quality documentation won’t have undocumented codebases to begin with.
And let’s face it, like writing code, writing docs does have a lot of repetitive, boring, boilerplate work, which I bet is exactly why it doesn’t get done. If an LLM is filling out your API schema docs, then you get to spend more time on the stuff that’s actually interesting.
Doing either of them _well_ - the way you do when you actually care about them and they actually matter - is still so far beyond LLMs. Good documentation and good tests are such a differentiator.
A good example of the kind of result is something like the Laravel documentation[1] and its associated API reference[2]. I don't believe AI can help with this.
[0]: https://en.wikipedia.org/wiki/Docstring
If/when will this take over your job?
https://techcrunch.com/2025/04/29/microsoft-ceo-says-up-to-3...
I get paid for the mundane, boring, annoying tasks, and I really like getting paid.
Even in the early days of LLM-assisted coding tools, I already know that there will be executives who would said: Let's replace our pool of expensive engineers with a less expensive license. But the only factor that led to this decision is cost comparison. Not quality, not maintenance load, and very much not customer satisfaction.
We started with Pro+ and Enterprise first because of the higher number of premium requests included with the monthly subscription.
Whilst we've seen great results within GitHub, we know that Copilot won't get it right every time, and a higher allowance of free usage means that a user can play around and experiment, rather than running out of credits quickly and getting discouraged.
We do expect to open this up to Pro and Business subscribers - and we're also looking at how we can extend access to open source maintainers like yourself.
That ensures that all of Copilot's code goes through our normal review process which requires a review from an independent human.
Pretty much every developer at GitHub is using Copilot in their day to work, so its influence touches virtually every code change we make ;)
Where does the most come from? There's a certain sense of satisfaction in knowing I've tested a piece of code per my experience in the domain coupled with knowledge of where we'll likely be in six months. The same can be said for documentation - hell, on some of the projects I've worked on we've entire teams dedicated to it, and on a complicated project where you're integrating software from multiple vendors the costs of getting it wrong can be astronomical. I'm sorry you feel this way.
Actually if you want well-written prose you'll read AI slop there too. I saw people comparing their "vibe writing" workflows for their "books" on here the other day. Nothing is to be spared, apparently
Thats a fun stat! Are humans in the #1-4 slots? Its hard to know what processes are automated (300 repos sounds like a lot of repos!).
Thank you for sharing the numbers you can. Every time a product launch is announced, I feel like its a gleeful announcement of a decrease of my usefulness. I've got imposter syndrome enough, perhaps Microsoft might want to speak to the developer community and let us know what they see happening? Right now its mostly the pink slips that are doing the speaking.
By that logic, literally every statement would be survivorship bias.
Ah yes, the takeoff.
Genuine question, but is CoPilot use not required at GitHub? I'm not trying to be glib or combative, just asking based on Microsoft's current product trajectory and other big companies (e.g. Shopify) forcing their devs to use AI and scoring their performance reviews based on AI use.
We absolutely have not reached anything resembling anyone's definition of a singularity, so you are very much still not proven correct in this. Unless there are weaker definitions of that than I realised?
I think you'll be proven wrong about the economy too, but only time will tell there.
I would not be surprised if things end up the other way around – humans doing the boring and annoying tasks that are too hard for AI, and AI doing the fun easy stuff ;-)
I expect though that most people don't read in that much detail, and AI generated stuff will be 80-90% "good enough", at least the same if not better than someone who doesn't actually like writing documentation.
> What is the job for the developer now? Writing tickets and reviewing low quality PRs? Isn't that the most boring and mundane job in the world?
Isn't that already the case for a lot of software development? If it's boring and mundane, an AI can do it too so you can focus on more difficult or higher level issues.
Of course, the danger is that, just like with other automated PRs like dependency updates, people trust the systems and become flippant about it.
I'd argue the vast majority of software development is neither critical nor commonly used. Anecdotal, but I've written documentation and never got any feedback on it (whether it's good or bad), which implies it's not read or the quality doesn't matter.
They just cut down their workforce, letting some of their AI people go. So, I assume there isn't that much success.
Copilot said: There is currently no official GitHub setting or option to remove or hide the sidebar with "Latest Changes" and similar widgets from your GitHub home page.
I'm using this an example to show that it is no longer possible to set up a GitHub account to NOT use CoPilot, even if it just lurks in the corner of every page waiting to offer a suggestion. Like many A.I. features it's there, whether you want to use it or not, without an option to disable.
So I'm suss of the "pretty much every developer" claim, no offense.
Similarly, the newest MS Word has CoPilot that you "don't have to use" but you still have to put up with the "what would you like to write today?" prompt request at the start of every document or worse "You look like you're trying to write a...formal letter...here are some suggestions."
Have they tried dogfooding their dogshit little tool called Teams in the last few years? Cause if that's what their "famed" dogfooding gets us, I'm terrified to see what lays in wait with copilot.
I guess maybe different teams have different requirements/workflows?
As part of the dogfooding I could see them really pushing hard to try having agents make and merge PRs, at which point the data is tainted and you don't know if the 1,000 PRs were created or merged to meet demand or because devs genuinely found it useful and accurate.
If we're expected to even partially believe the marketing, LLM coding agents are useful today at junior level developer tasks and improving quickly enough that senior tasks will be doable soon too. How do you convince so many junior and senior level devs to build that?
LLM use is now part of the annual review process, its self reported if I'm not mistaken but at least at Microsoft they would have plenty of data to know how often you use the tools.
It’s very, very far from possible today.
We have invested plenty of money and time into nuclear fusion with little progress. The list of key acheivments from CERN[1] is also meager in comparison to the investment put in, especially if you consider their ultimate goal to ultimately be towards applying research to more than just theory.
I'm interested in the [vague] ratio of {internallyDevlopedTool} vs alternatives - essentially the "preference" score for internal tools (accounting for the natural bias towards ones own agent for testing/QA/data purposes). Any data, however vague is necessary, would be great.
(and if anybody has similar data for _any_ company developing their own agent, please shout out).
The goal here is for it to be able to do everything, taking 100% of the work
2nd best is to do the hard, big value adds so companies can hire cheap labor for the boring shit
3rd best is to only do the mundane and boring stuff
After hearing feedback from the community, we’re planning to share more on the GitHub Blog about how we’re using Copilot coding agent at GitHub. Watch this space!
Without data, a comprehensive study and peers review, it's a hell no. Would GitHub willing to be at academic scrutiny to prove it?
It seems places don't prioritize it, so you don't see it very often. Some developers are outright dismissive of the practice.
Unfortunately, AI won't seemingly help with that
> There's a certain sense of satisfaction in knowing I've tested a piece of code per my experience in the domain coupled with knowledge of where we'll likely be in six months.
i suppose its possible at some point an ai could be complex enough to try out additional edge cases or confirm with a design document or something and do those parts as well... but idk its still after-the-fact testing instead of at design-time its less valuable imo...
I can imagine there are groups where it is true. I mostly want to push back on the idea that there's one monolithic culture that is Microsoft.