Android shouldn't be considered Open Source anymore, since source code is published in batches and only part of the system is open, with more and more apps going behind the Google ecosystem itself.
Maybe it's time for a third large phone OS, whether it comes from China getting fed up with the US and Google's shenanigans (Huawei has HarmonyOS but it's not open) or some "GNU/Linux" touch version that has a serious ecosystem. Especially when more and more apps and services are "mobile-first" or "mobile-only" like banking.
Allowing apps to say "we only run on Google's officially certified unmodified Android devices" and tightly restricting which devices are certified is the part that makes changes like this deeply problematic. Without that, non-Google Android versions are on a fair playing field; if you don't like their rules, you can install Graphene or other alternatives with no downside. With Play Integrity & attestation though you're always living with the risk of being cut off from some essential app (like your bank) that suddenly becomes "Google-Android-Only".
If Play Integrity went away, I'd be much more OK with Google adding restrictions like this - opt in if you like, use alternatives if you don't, and let's see what the market actually wants.
There are a lot of scams targeting vulnerable people and these days attacking the phone is a very "easy" way of doing this.
Now perhaps there is a more forgiving way of implementing it though. So your phone can switch between trusted and "open" mode. But realistically I don't think the demand is big enough for that to actually matter.
Even with play integrity, you should not trust the client. Devices can still be compromised, there are still phony bank apps, there are still keyloggers, etc.
With the Web, things like banks are sort of forced to design apps that do not rely on client trust. With something like play integrity, they might not be. That's a big problem.
Play integrity hugely reduces brute force and compromised device attacks. Yes, it does not eliminate either, but security is a game of statistics because there is rarely a verifiably perfect solution in complex systems.
For most large public apps, the vast majority of signin attempts are malicious. And the vast majority of successful attacks come from non-attested platforms like desktop web. Attestation is a valuable tool here.
As for compromised devices, assuming you mean an evil maid, Android already implements secure boot, forcing a complete data wipe when breaking the chain of trust. I think the number of scary warnings is already more than enough to deter a clueless "average user" and there are easier ways to fish the user.
Brute force attacks on passwords generally cannot be stopped by any kind of server-side logic anymore, and that became the case more than 15 years ago. Sophisticated server-side rate limiting is necessary in a modern login system but it's not sufficient. The reason is that there are attackers who come pre-armed with lists of hacked or phished passwords and botnets of >1M nodes. So from the server side an attack looks like this: an IP that doesn't appear anywhere in your logs suddenly submits two or three login attempts, against unique accounts that log in from the same region as that IP is in, and the password is correct maybe 25%-75% of the time. Then the IP goes dormant and you never hear from it again. You can't block such behavior without unworkable numbers of false positives, yet in aggregate the botnet can work through maybe a million accounts per day, every day, without end.
What does work is investigating the app doing the logging in. Attackers are often CPU and RAM constrained because the botnet is just a set of tiny HTTP proxies running on hacked IoT devices. The actual compute is happening elsewhere. The ideal situation from an attacker's perspective is a site that is only using server side rate limiting. They write a nice async bot that can have tens of thousands of HTTP requests in flight simultaneously on the developer's desktop which just POSTs some strings to the server to get what they want (money, sending emails, whatever).
Step up the level of device attestation and now it gets much, much harder for them. In the limit they cannot beat the remote attestation scheme, and are forced to buy and rack large numbers of genuine devices and program robotic fingers to poke the screens. As you can see, the step-up from "hacking a script in your apartment in Belarus" to "build a warehouse full of robots" is very large. And because they are using devices controlled by their adversaries at that point, there's lots of new signals available to catch them that they might not be able to fix or know about.
The browser sandbox means you can't push it that far on the web, which is why high value targets like banks require the web app to be paired with a mobile app to log in. But you can still do a lot. Google's websites generate millions of random encrypted programs per second that run inside a little virtual machine implemented in Javascript, which force attackers to use a browser and then look for signs of browser automation. I don't know how well it works these days, but they still use it, and back when I introduced it (20% time project) it worked very well because spammers had never seen anything like it. They didn't know how to beat it and mostly just went off to harass competitors instead.
How is the attacker supposed to bruteforce anything with 2-3 login attempts?
Even if 1M node submitted 10 login attempts per hour, they would just be able to try 7 billion passwords per month per account, that's ridiculously low to bruteforce even moderately secure passwords (let alone that there's definitely something to do on the back end side of things if you see one particular account with 1 million login attempts in a hour from different IPs…).
So I must have misunderstood the threat model…
The threat model is entirely different from what your brute force phrase implies, and it is also a threat model that isn't relevant to banking, which was the topic of the discussion in the first place. And more importantly, it doesn't affect the security of the user.