Skip to content

Decide and document how logs treat user data #3975

Open

Description

@pbleser-oc

OpenCloud's structured logs contain user data where it helps to understand a problem: user names, email addresses, file names and paths, tags, LDAP DNs, error messages of backends. Secrets such as passwords, keys and tokens are never logged, and that doesn't change. For user data, this works, but it isn't a strategy the services follow consistently, and it isn't documented anywhere. This issue is about agreeing on one approach and writing it down, so that sysadmins know what their logs contain and whether they need to do anything about it.

Current state

  • Each service decides on its own which user data it logs, and under which key names (see Use globally defined log attribute key names across all services and packages #3290 for the key names).
  • The documentation doesn't say which kinds of user data can end up in the logs. A sysadmin who has to decide on access to the logs, their retention, or shipping them to an external log service, has no reference to base that on.

Trade-offs

Logging user data has real benefits:

  • Admins can find the user, file or space affected by a problem directly in the logs, without having to reproduce it.
  • Log entries can be matched with support requests ("user X couldn't upload file Y").

It also has costs:

  • Logs that contain user data are personal data themselves, which matters for access rights, retention and log shipping.
  • The more services log such values, each in its own way, the harder it is to answer "what is in our logs?" completely.

Neither side is clearly right for every installation, which is why a stated and documented approach matters more than the choice itself.

What to decide

  1. Which kinds of user data may appear in logs. The documentation should also state the existing rule that secrets (passwords, keys, tokens) never do.
  2. Shared, semantic key names for these values (paths, user names, email addresses, tags, ...), so that every service logs them the same way. This builds on Use globally defined log attribute key names across all services and packages #3290.
  3. Whether to offer an optional way to protect these values. Since the logs are structured, a logging middleware could hash or mask the values of these keys in one place, instead of every service handling it on its own. With hashing, entries of the same user or file can still be matched with each other.
  4. Where to document it for sysadmins: what the logs may contain, and what they can do if that doesn't suit their installation.

A possible order: agree on the key names first, make all services follow them, then decide on the optional middleware. Until then, the services keep logging as they do today.

Related: #3290 (log key names), #3301 (length of logged values).

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions