Among the text-to-image services for obtaining a 360º AI generated image, Skybox AI by Blockadelabs is the most reliable and the one that allow us to enhance the prompt by feeding the AI with an equirectangular sketch map.

This Latent Diffusion Model for 3D (LDM3D), proposed by Stan et al. (2023), generates both a 360º RGB image and depth map data from a given text prompt. The LDM3D model is fine-tuned on a dataset of tuples containing an RGB image, depth map and caption, and validated through extensive experiments.

Based on the panoramas sketched on location, I stylized the equirectangular sketch map with the main elements of the composition.

It must be white strokes on black background. The first results are promising: the generated image is adherent to the main composition, taking details from the prompt and the style from the parameters.

There is still a lot of research to be done on how to manage the lines, the amount of details and possible filled areas. For the purposes of this installation, the process works well.

References
Skybox AI. (n.d.). Retrieved January 23, 2024, from https://skybox.blockadelabs.com
Stan, G. B. M., Wofk, D., Fox, S., Redden, A., Saxton, W., Yu, J., Aflalo, E., Tseng, S.-Y., Nonato, F., Muller, M., & Lal, V. (2023). LDM3D: Latent Diffusion Model for 3D. https://doi.org/10.48550/ARXIV.2305.10853