To avoid the typical picture on the background of your desktop, use the following procedure to force a solid blue color as background:

• Start the Group Policy Editor: gpedit.msc
• Go to User Configuration > Administrative Templates > Desktop > Desktop
  
• Open Desktop Wallpaper
  
• Enable it and set the Wallpaper Name to:
  C:\Windows\Web\Screen\img105.jpg
• Click on OK to save the changes

While you could also just right-click on the desktop and choose Personalize and set a solid background color, this did not stick on my Windows 2022 Servers (it always returned to a black background after a while). The method above was the only one to keep the blue background permanently.

Side-note: If you want the same background on the lock screen and during logon as well, follow this procedure:

Windows Server: How to force a certain Lock and Logon Screen