Towards Sustainability Model Cards: A DSL for specifying AI’s Environmental Impact

As AI models grow larger and more complex, their environmental footprint becomes increasingly concerning. During the model training phase but also when running them as each inference (see the AI Energy score leaderboard). Yet, when we choose which AI model to use for our applications, sustainability metrics are difficult to find. Especially in a kind of a standard format that could also enable an automatic selection process based on (non-functional) requirements.

In our paper Towards Sustainability Model Cards we aim to leverage the concept of quality models and service level agreements (SLAs) common in other fields and merge them with existing ML model reporting initiatives and Green/Frugal AI proposals to formalize a Sustainable Quality Model for AI/ML models.

As a first step, we propose a new Domain-Specific Language to precisely define the sustainability aspects of an ML model (including the energy costs for its different tasks). This information can then be exported as an extended version of the well-known Model Cards initiative while, at the same time, being formal enough to be input of any other model description automatic process.

If you’re interested in the full technical details, you can read the full paper here. The work will be presented at the 2nd Workshop on Green-Aware Artificial Intelligence in Bologna, Italy.

What Are Sustainability Model Cards?

Sustainability Model Cards extend the traditional Model Cards concept by adding comprehensive environmental impact information. The proposed framework organizes this information into four main sections:

1. Metadata. This section captures basic model identification including name, version, type (CNN, transformer, etc.), provider, and license. This links the sustainability card to existing model documentation, enabling combined analysis of performance and environmental metrics.
2. Training Information. . The training section documents not just energy consumption and carbon emissions, but also water consumption. The framework captures training duration, the platform used, and quantified environmental impacts with proper units.
3. Inference Metrics, Different tasks have different environmental costs. A model might be efficient for text classification but resource-intensive for text generation. The framework allows documenting multiple inference tasks, each with its own environmental metrics, helping users choose the right model for their specific use case.
4. Platform Details. Infrastructure matters. The same model running in a region powered by renewable energy has a different carbon footprint than one running on fossil fuels. The platform section captures hardware specifications, cloud provider details, regional deployment information, carbon offset credits, and the energy mix used.

A Domain-Specific Language for Formal Specification

What sets this work apart from existing sustainability reporting efforts is the introduction of a precise DSL to define the sustainability information of a model. This isn’t just about documentation—it’s about creating machine-processable specifications that can be integrated into automated workflows.

As usual, the DSL consists of two main components:

Abstract Syntax

A metamodel that defines the structure and relationships of sustainability concepts. There are three main subcomponents: MetaData, Training and Inference. The full definition of the metamodel can be seen in the following figure.

 

 

Concrete Syntax

A YAML-based notation that makes the cards both human-readable and machine-processable. Here’s a simplified example of what a sustainability card looks like:

sustainability_model_card:
  meta_data:
    name: Model Name
    version: 1.0.0
    model_type: LLM
    provider: Provider Name
    license: CC0
    
  training:
    hour_duration: 100.0
    platform: Infrastructure
    carbon_emissions:
      value: 14.25
      unit: kgCO2eq
    energy_consumption:
      value: 25.0
      unit: kWh
    water_consumption:
      value: 57.5
      unit: L
      
  inference:
    - task:
        inference_type: TextGeneration
        platform: Infrastructure
        carbon_emissions:
          value: 7.12
          unit: gCO2eq
        energy_consumption:
          value: 12.3
          unit: Wh

The YAML format was chosen deliberately to integrate seamlessly with existing Hugging Face model cards, lowering the adoption barrier for the ML community.

Tool Support

We provide a Python implementation of our DSL that includes a validating parser and classes implementing the metamodel. The implementation leverages the BESSER Low-Code platform and is available on GitHub.

The parser performs several validation checks to ensure data quality:

• Verifying that all measurements include appropriate units
• Ensuring units correspond to those defined in the metamodel
• Validating that percentage values fall within the [0,1] interval
• Checking that inference and energy types match predefined categories

Research Roadmap

This is just a first step in a more ambitious goal mixing DSLs and model-driven engineering to power automatic treatment of sustainability concerns in AI. Some aspects we plan to keep working on are:

Extending Coverage: Future versions could include more granular training information (distinguishing pre-training from fine-tuning), hyperparameter configurations, and dataset specifications. As the Green AI field evolves, new metrics and frameworks will need to be incorporated.

Multiple Concrete Syntaxes: While YAML works well for technical users, the framework could benefit from graphical notations for less technical stakeholders or even conversational interfaces for card creation and querying.

Integration with Existing Standards: Tighter integration with traditional Model Cards would provide a complete view combining performance, ethical, and environmental considerations. The formal DSL foundation makes this integration technically feasible. Indeed, an interesting direction would be to extend the sustainability concern with ethical concern to allow more transparency in the ethical and environmental sustainability impact of AI models.

User Impact Studies: Understanding how sustainability information actually influences model selection decisions will be crucial. Do users prioritize accuracy over emissions? How much accuracy are they willing to sacrifice for sustainability? These questions need empirical investigation.