Allow silencing Prometheus alerts directly from Slack (and other sinks)

[aantn]

Motivation
Sometimes alerts are really noisy and you just want to silence them temporarily. You can do so by opening AlertManager and creating a silence in the UI (or by using the amtool cli) but you can't do so directly from Slack.

Suggested Solution

1. Write a playbook action which takes a Prometheus alert and silences it
2. Write a playbook action which is triggered on every Prometheus alert and adds a CallbackBlock which lets the user trigger (1) when they choose

[arikalon1]

@aantn Can we close it, since we implemented the alert silencing? Or you had something else in mind here

[aantn]

@arikalon1 closing, that was exactly what I had in mind and we've implemented it now.