← Back to Blog
Saturday, January 21 2023

How To Handle High Volumes Crypto Mining Attacks while still offering a Free Plan (Compute)

`Cryptojacking` or malicious crypto mining is a security threat in which attackers abuse compute resources to mine cryptocurrency without the organization’s consent.

Noteable is a collaborative data notebook that enables data-driven teams to use and visualize data, together. Doing so, we offer a free plan on our multi-tenant with unlimited compute. There is no gotcha and no limit in terms of product usage. We became attractive to malicious actors who wanted to run scripts on Noteable for different reasons, such as Cryptojacking. We are sharing how cryptojacking threats can be significantly reduced by implementing various security measures depending on the organization’s needs. 

Cryptojacking or malicious crypto mining is a security threat in which attackers abuse compute resources to mine cryptocurrency without the organization’s consent. This can impact performance, increase business costs, and generate profits for malicious users through the mined cryptocurrency.

Due to the nature of our product and the kernels allowing arbitrary code execution, Noteable quickly became a target for cryptojacking attacks. In the hope of helping other companies to deal with these, we are sharing our journey that walks through how we:

  • detect malicious activity,
  • identify the malicious patterns and;
  • mitigate existing and future cryptojacking activity

We leveraged our own product to manage the complete incident response ranging from digging into the logs to acting and blocking accounts using Auth0 APIs.

Detecting Cryptojacking

One of the immediate signals for us was the increase in CPU-intensive tasks and a spike in cloud computing costs. In general, both are significant indicators to monitor continuously for early detection of a potential attack.

We built a Noteable notebook (another example of a public notebook) to analyze data collected from cloud billing reports. We could not share the actual notebook due to the confidentiality of the data, but we are sharing some visualizations. It helped us to detect cost patterns and show recent anomalies quickly.

iadRs 7whKRrdfuK0xtX6st tiKtsZlgN9UMhK8jV0scvd1ezYmbDJF Yg6mOH4RtfDLAn04 RSYb04tT7NrcZfBgdVMjOKVXUNmhl4J5ZsHGfAEac4I HIlhWjqDybQ07I56DFuj2O6850dzE7iSP9Zo0Lslpwrw7pPGccQiEkSc556zMmIcJ4HrB6G8Q

Leveraging our interactive data explorer, within the first hour of the incident, we had a clear picture of the scope of the issue. In addition, our collaboration functionality could bring the entire team on the same page. 

How bad can it get?

We started with some occasional occurrences and just a couple of detected cryptominers; it quickly evolved to a more complex and hard-to-manage scenario with multiple engineers involved across DevOps and Backend and an effort required to track down and kill crypto mining processes.

LWiDBIH4v7ZE7VITV5Z36aT2iIzl7kTb0i9Lmr2gLA8ZKvfNqK0mEwnWxRFBO1Rml5QXOmUGz8DDrOQRLvEqFJakacz5ESkRvH4OnwnWaCKl 6mgTcwG9MuKU9ekzX8G026lXfcm bxmLO GtMme0tEaVw1CNK P2sN9LmfBLgbLrBCQLynjBk8JsccBgw
How To Handle High Volumes Crypto Mining Attacks while still offering a Free Plan (Compute) 8

Noteable secrets allowed a smooth integration with Auth0 API to retrieve a list of cryptominers and their locations. The below tile map visualization shows the cryptominers distribution around the world.

vCvg7BmiAstutHN5P4r6f rL 4wjtyYuWhyJ9kCjXnbB2q784ZjqOnBxu6FACYj1KGVRf2weRCLchJZ3XS5L8 YJhBYx2Em 2KP407gUHzSvzIFzO Nxag6L6RvTiilc sT3v fp8AyAl s1sMLEAcHEyfb5a82 FHQM8J7j84O

How Noteable implemented security measures for cryptojacking

  • We followed the best practices for securing Kubernetes platform. Firewalls and network policies are configured to prevent unauthorized communications.
  • We put in place resource usage monitors to detect potential crypto mining activity. Crypto mining requires a lot of processing power and can consume a significant amount of resources, such as CPU, memory, and network bandwidth. If there is an unusual spike in resource usage, it could be a sign of crypto mining activity.
  • We scheduled our noteable notebooks to run daily monitoring compute costs, usage and detect spikes as potential indicators of cryptojacking. 
  • We integrated Sysdig Secure for detection. Sysdig Secure deploys different types of policies for threat detection, including access to crypto mining networks, suspicious crypto mining rules, and machine learning policies to detect cryptominers. It is possible to control what should happen to affected containers if the policy rules are breached in an automated way.
  • We implemented Seon fraud prevention tools. Seon evaluates risks and helps to detect malicious accounts during the sign-up process. It is a powerful tool to identify fraudulent behavior patterns with scoring engine features.

Below we show the rules’ effectiveness over time. Customized parameters and increased fraud scores rejected a higher number of crypto miners.

YBe80EkuzoVFij2ErGKV9Lw 2s8rtUuoCQxXrAsVmFcFYA3FMM8YCahq6285kQYKTExo29MWvwUUCqPbE2NfOqt5sqrWwnQvTzClwdQ6YH4foBQ6ycd5KVKIoFjfzX92RAP d0FWvuBrI8ERnm968pNSnD1a4rBroan5JJPA9TnA2v 1eB63hwoeczTBA

We can also look at the applied rule statistics and recurrence:

noSwbow9ek9nxo CFYgf0odvr7Ed7KX5K6OBYS6gthoBRPVINwc29vr555hTDBMgDJji8BlHwMzrNMKTbTbZ7G38PyJYBorEquGY22cghPjw4KpS q2Pa7KzN0sQRdalmZPlj5B5nztDcxBirjZGX 6QJGoEmE4OwGzgT SwfdLB6pSyh37fmuxfj CksA

Finally, we set up notification channels whenever thresholds and violations occurred for broader visibility. We have slack notifications from our notebooks to our slack channels. We leverage Secrets to store Slack credentials, and a Python code cell sends the notification. It is super easy, and we can have conditional notifications whenever specific criteria are met.

The workflow above gives us enough feedback loop and data to update our rules to improve detection continuously. In security, observation and fine-tuning configurations are key. Attackers live to come up with ways to abuse while we live to find that new way as soon as possible!

Our most recent data shows an outstanding decrease in attacks:

FEPA0uykWSnnXtPMok88WIWOSDK0fKn5NB 6oJ4s7NqF5jXQUyTY5QP4ZHDoQh7QBK3A lEILE6sbX7XJe WijcKqha69pINz5BkLAOYvn3FsyvYwp4kdDYBd03BtSBfnwTEm0KoGMySQElvCsjbvNVnC ncj2PYbU VTXkSnFlcfyt19J1VN2K0iIyXWw
lDYCCvehbXLcdMa87eFiLfZU8QSweFTQGe3TX 9ci8HWOnimuwnSDlIe26G7PrIhNPWIMsBwwloDFzzFKWOM8vppux13QmfgjQ naywC7ALs756PsSJCxtd

Long-term mitigations require more than putting a process in place. It’s an ongoing effort to stay up to date with the latest security events in our infrastructure and the implementation of effective policies. Using Noteable no-code visualizations capability simplifies the fight against cryptojacking by reducing time to insights to take quick action and customize measures based on crypto mining patterns. Aggregating data from different sources, running Python or SQL code, and then visualizing the results took less than a day of work. Most importantly, having the entire team be able to collaborate made the incident response process streamlined, as it was a single document that the entire team was using to orchestrate multiple prongs of the response.

We would love to connect if you want to learn more about our ongoing effort to tackle and mitigate malicious attacks.