The Watchtower Project

What is Watchtower?

Watchtower is a research project to scan an unprecedented volume of Internet material for sensitive credentials, like cloud access keys.

It works by scanning a variety of content formats for patterns associated with secrets. For example, it not only looks at publicly-facing pages, but also mobile and executable apps.

I received an email about my key being leaked, what happened?

You likely found this site after receiving an email from your cloud provider regarding a leaked secret and are seeking clarity.

When Watchtower encounters potential credentials, it verifies them against the provider and if legitimate, leverages the GitHub secret scanning platform to notify the cloud provider about the leak. Remember, if we were able to see your key, so could attackers.

Usage of your key by this project was strictly limited to 1) validating that it is not a false positive, and 2), using GitHub to promptly secure it. To trigger GitHub's mechanisms, we create a private note with the key in it. No one but us can see the key when we do this. Of course, if an attacker found the original location of the leak, they could still access the key material.

The reason we did not report this issue directly is because it is nearly impossible to automatically determine an impacted organization based off an access key, without manual investigation. This is why we use GitHub- we want to avoid exposing your organization to any more risk than necessary.

What should I do?

Depending on the provider, your key may have been revoked, isolated, or simply issued a warning. We would strongly recommend that if it is not done already, you revoke the key to prevent abuse. It is never safe to have the credentials for cloud infrastructure exposed. Even if the key has an incredibly limited scope, one mistake could lead to a significant compromise.

It's important to have good security hygiene through policies for how keys & secrets are used. Examples include:

• Never embed secrets in a source file directly. Always store them in a safe place, like a secure key vault, with appropriate access controls.
• For development use cases, environment variables are acceptable but still pose a higher threat than a key vault.
• Client facing applications should almost never need direct secret access. You should use secrets in a secure backend that exposes a limited surface for your application's requirements.
• Rotate keys on a scheduled basis. Enforce reasonable expiry windows for secrets. This helps reduce the risk of abuse if it is leaked anyway.

More important than following these requirements in general is having controls to enforce them. You are only as secure as your weakest link. Applying these practices consistently across your organization is paramount.

Can you tell me where my key was leaked?

Watchtower deals with an unprecedented volume of secret material. You are one of many organizations that it has helped protect. Unfortunately, due to the limited data it stores to reduce the already extremely high infrastructure costs from operating at this scale, it would be cost prohibitive to perform such a search for each.

We often see keys embedded in client-facing executables or scripts (JS) on websites. We'd recommend that you investigate the uses of this key within your organization to determine an origin.

Who are you?

My name is Bill Demirkapi and I am an independent security researcher working in the information security industry.

This is one of many projects I run on my own time, completely independent from my employer, to help make the Internet safer.

While I can't guarantee a response due to my limited time, if you'd like to get in touch or learn more about me:

email: billdemirkapi (AT) gmail (DOT) com
personal research blog: https://billdemirkapi.me/
twitter (dms are open): https://twitter.com/BillDemirkapi