Skip to content

Provide a visualization for the norms of different parts of a transformer model #25

Description

@iislucas

A common debugging method is to monitor the norm of some part of a ML model's weight and activation space.

Goal:

  • Create a visualization component for norms of a set of vectors and/or weights. Allow the norms to be tracked over time, like we do with loss/accuracy.
  • The code and implementation should be reusable and clean.

Code pointers:

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestproject:TTPFor the Tiny Transformer Playground

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions