[bghira]

This pull request adds new high-memory Cosmos3 image training examples and updates existing example configurations to use improved attention targeting and PyTorch Dynamo settings. The main changes include the introduction of 48GB and 80GB GPU-specific configs, consistent use of Cosmos3PackedMoTAttention in LyCORIS configs, and the addition of a new dataset backend for 1024px image training.

New high-memory Cosmos3 image examples:

• Added cosmos3-image-48g.lycoris-lokr and cosmos3-image-80g.lycoris-lokr example directories, each with tailored config.json and lycoris_config.json files for 48GB and 80GB GPUs, respectively. These configs include settings for ARB 1024px training, gradient checkpointing (48G), and FlashAttention 3 (80G). [1] [2] [3]

• Added a new dataset backend config, multidatabackend-cosmos3-domokun-1024-arb.json, for 1024px image training with the RareConcepts/Domokun dataset.

Configuration improvements for all Cosmos3 examples:

• Updated all relevant lycoris_config.json files to target Cosmos3PackedMoTAttention instead of the generic Attention module, ensuring correct application of LoRA/LoKR factors.

• Enabled PyTorch Dynamo with inductor backend and regional compilation in all example config.json files for improved performance. [1] [2] [3] [4]

Documentation:

• Updated the README.md to describe the new high-memory Cosmos3 image examples and their specific hardware/feature targets.