You can't, but you can play a variant that requires a lower amount of players. For instance, the Ancient Meditteranean variant only requires 5 players.

If that is not what you want, I recommend to use our sister site, vDiplomacy. It has a huge number of variants, for varying numbers of players, that are playable. On https://vdiplomacy.net/variants.php you can see all the different variants. You can sort them by player number. Also, there is the variant 'Classic - Pick your countries' (https://vdiplomacy.net/variants.php?variantID=42), in which you play the classic map, but can choose which countries are playable. Make sure to check the variant description before setting up a game though. In the case of 'Classic - Pick your countries', the name you give to your game codes for the specific countries that will be playable.

I am looking how to set a diplomacy game to a lower amount of players.