In case you missed it, we released support for .NET 6 Preview 4 about 2 weeks ago.
Any app targeting the .NET 6 Early Access on App Service will be automatically updated to the latest .NET 6 Preview releases as they become available all the way up to RC and GA.
If you are creating a new .NET 6 app on App Service today, you will have Preview 5 available on the platform. If your app was created earlier this week, it will be automatically updated to Preview 5 the next time it cold starts. If you want to trigger the update, you just start/stop the app.