I've seen this conversation before. The best example that I can think of is Jackson Holliday; a player rakes in Spring Training (in Holliday's case, batting .311 with 3 doubles, 2 triples and 2 homers in 15 spring games in 2024) and then getting left off the Opening Day roster for more development.
The annoying thing about Holliday is that he was the top prospect going in to 2024 -- and in the minor league season of 2023 he batted .323/.442/.499 with 113 runs, 30 doubles, and 24 swiped bags in 125 games.
Holliday was evidently not ready for the majors -- in 60 MLB games in 2024 he batted just .190 with a .565 OPS. Though I am of complete confidence that Holliday will be fine and even a star of the game in the near future, his case represents why a team would keep a player down even if he hits well in spring.
Jac Caglianone's case is a little bit different. He was drafted in the first round of the 2024 draft by the Kansas City Royals. The highest level he ever played was High-A ball, where he slashed .236/.302/.388.
However, in 2025's Spring Training, Caglianone is batting .500/.609/1.167 with 3 homers in 14 games. This is nothing shy of amazing. Take a look at this beautiful third home run of Caglianone's spring: