A lot of people are just looking at those Tflops, but the fact that the PS5's SSD is twice as fast and that the chipset can handle those speeds will make PS5 a fierce competitor to the XSX.
Thing is a lot of games are developed as multi platform projects. This current generation was a clear win for the PS4 because while both consoles are reasonably close to a PC, PS4 is the faster 'gaming PC' out of the two (not counting mid gen refreshes) and had a better start, hence Xbox suffered.
If a console goes back to being a special setup with unique capabilities then multi platform titles are less likely to get tweaked for it. PS3 was difficult this way, PS2 and previous equally so. So you end up with titles targeted at the platform, making use of particular features in a way you couldn't elsewhere but your EA/Ubi/Actibliz/whathaveyou titles might just be targeted at PC/X and ported to the Playstation, potentially making that the worst performing ports of their titles.
I'm not playing those multi plats myself so I actually do care little about it but apparently Fifa, Modern Warfare, Rockstar titles and the like are the big sellers way beyond Sony's inhouse productions.
Of course if the fast SSD can make the difference in a way that is straight up advantageous not just in special circumstances and simply enhance things then it may all work out well for PS. But if the CPU and in particular the GPU is a good chunk weaker like what the numbers suggest then that's a worry considering all this stuff is being developed on PC and also with an Xbox version in mind.
A PS5 with a unique architecture would have to pull impressive numbers right from the start to make that the lead platform.