September 13, 2021

Getting Apollo.io users password

Bad digits injection

Three months ago, I noticed a security issue on the Apollo website. I tried to inform them about this (including that at some point I could publicly disclose it), and I had absolutely no reply from their support team. I decided to publicly tell the story and share the details. I'm sure they will fix in hours, the goal to inform them to fix this is about to be reached in a more effective way than before. Now it's more than 90 days after the initial notice. I'm sharing with you some more details. The issue is still present. Edit : seems to be fixed end of 2022, using the method "no need to display the email back".

Tune the Apollo's strings

How I discovered the issue ? I shared an email in one of their web form. And the next web page displayed a message like "Thanks you for registering your email emailaddr@domain.com", and the email was given in the page address URL link. So I imagined that there's maybe a security issue if they made a mistake with this point, and I started to look at the possibility of injection of my own code through the given email address in the address. The first try was to inject JavaScript <script> alert but it was detected and blocked from the Web Application Firewall from CloudFlare they are using. First try is a failure, there's a security measure to prevent this, so what are the other ways to inject data? I thought of some obfuscations techniques to hide the JavaScript code and bypass the firewall. But I wanted to take a very different approach: inject HTML code instead of the JavaScript. And that works! <b>Hello!<b> in the URL bar is printed as bold text that I just chose in the Apollo web page.

A first proof of concept, very interesting! Notice the text in italic from the <i> tag. That means my data in args is interpreted as HTML markup. The data text I provide in the URL was added in the html page code, this injects my own markup in the page content.

But how can this be a real issue if I'm just reading back on my own computer what I just sent? Well, a bad actor can forge an URL, sends it the to the victims, and get the account email and password, we'll see how.

I imagined how this issue of "reflected" injection can be turned into a real security issue, which can lead to steal the Apollo's user password. How can we steal password with HTML only, no JavaScript, no script at all? That's the title of this post after all. First, I will explain this kind of issue in a generic way. Eventually, some web developers would read this actual story and will be aware of this issue, so not doing it in their own code. I'm kind, and at the end I provide many security measures one can take to prevent these kind of issues.

XSS?? Is that safe for work?

This issue is what we call an XSS vulnerability : Cross Site Scripting. This is when a third-party can inject scripts or code in a target "trusted" website. In our case, the trusted website is the Apollo website www.apollo.io, and we can inject HTML code into one of these webpages. That means that anyone can add or modify the HTML code of a web page on this site. The issue with XSS is that it uses the trusted domain of the website. Here, the browser displays to the users the Apollo domain, with their own "green lock". So the user has not many ways to see the attack and she thinks she's on the Apollo website. And actually it is the legitimate website, it's just that we can inject some external code that is executed/displayed by the web browser, like it is replacing the trusted website code with the malicious external code. The issue is present when some users input (here the email data in the URL) are not properly filtered, not sift enough, not "sanitized". And the data provided by a user (or a third party) is at some point later executed as code by the user (or others users).

There are several types of XSS, and this one is a self or reflected kind. Why self/reflected? Because the user inputs her own data, and this data is executed in her own web browser. I saw some bug bounty programs that explicitly excludes the self-XSS issues out of their security scope, no reward if you see and report a self-XSS issue. I feel this is like their care only about the security of their own server, the data in the server. And they totally don't care about the security of their users (web surfers). This is like in a physical shop, the security guard lets in people he knows to be a pick-pocket, because he knows the thief will only rob customers wallets, and he will not steal shop's employees nor rob the cash machine. So the security guard lets the thief proceed inside the shop, because according to the guard, it doesn't harm the security of the shop company. It's only a matter of security for customers inside the shop, and "it's not my business" the shop-owner think, it's the customer responsibility to protect themselves inside the shop, as they do outside. So here's my stance about the self-XSS exclusion policy. This kind of issues should be treated like any other security issues, with care, eventually be rewarded (in case of bugs bounty), and fixed promptly. Because a lack of data input sift in the company system code can cause some harm for the users (code execution, account data leak, phishing,...). I don't say Apollo is refuting that, I just saw some companies have written this as exclusion in their bug bounty program, and that's my point of view about this. Apollo, they just didn't reply to my support tickets alerting them.

So here's how this can lead to a security issue. Since the data comes from a URL, a malicious party can build the URL, and we are about to see that the data in the URL can be used to make a perfect phishing web site. Like a real one, because the domain is the actual one, not a fake copy. So this is not strictly speaking phishing. That's the issue, because you wouldn't have any doubt about the web page. The webpage is displayed as from the Apollo legitimate domain name www.apollo.io. And because of the injection, not all the page codes are coming from the Apollo servers, some HTML codes of the page can be provided from the URL arguments. As the dedicated OWASP web page explains : Reflected attacks are delivered to victims via another route, such as in an e-mail message, or on some other website. When a user is tricked into clicking on a malicious link, submitting a specially crafted form, the injected code travels to the vulnerable web site, which reflects the attack back to the user’s browser. The browser then executes the code because it came from a trusted server.

Let's practice XSS baby!

OK, enough theory. Let's proceed practicals. Some juddering strings will mesmerize the server, and defeat him.

  • Identify the vulnerable web page and mechanism, here it is simply : https://www.apollo.io/signup-success/?email=YOURHTMLCODE
  • Compose a nice HTML code to build a login form. The trick is with a HTML form, the browser sends to a given external server the data input in the page by the user. Without any script, pressing a button, triggers the data exfiltration. An example is provided here. And we can use the web page style class, so it's very easy to make something that looks exactly like their page style. Because it injects HTML and we got the same CSS as the current whole page. For example the "fake" login button has class button--oauth and it looks like a regular Apollo button (because in a way it is a real button from Apollo, thanks to XSS). Same trick for the font, and for all the presentation rendering styles.
  • Compress the HTML tags, removing line returns, spaces,... And then encode in "url-encoding", called "escape URL data" or "encodeURI". And fully encoding, all the text letters, like treating all characters as unsafe or reserved. So this will hide from the user suspicious words like "password", and it will be only gibberish. That gives something like that.
  • Add a very long chain of underscore "____..." to visually separate the domain in name from the long "payload" argument (sometimes call "searchpart"). This decreases the user awareness of the XSS attack, by hiding the complex payload, maximizing the chance to trick the victim.
  • You need to have a server online with an API endpoint that collects the data posted by the injected webform. I didn't go on this task, in the provided example, it uses acollect.server.testing.com as a dummy domain. So I didn't perform the whole attack. I just assessed its feasibility and I confirmed all the points, the browser tries to contact this dummy server with the login and password input. Also I don't know any Apollo user, not even my company, and I didn't test on real conditions.
  • Then you can send to a target victim, for example someone you know he's an Apollo user. Writing an email from the Apollo web service and stating that there's a $2'000 auto-renew feature activated, and there's a 48h short notice to deactivate this feature (else he will be charged this amount). And the link we just crafted is including in a [opt-out] button. So the victim user will probably rush to click on the email button and trigger the attack.

So how does it look like finally? What's the result? Here are some screenshots. The fake login modal is presented as an overlay to cover the page. So it really replaces the page central content, with the malicious one. Header is preserved to keep it real, and footer is rejected at the bottom with very high bottom margin.

Firefox example
Edge example

Pretty convincing, isn't it ? In Chrome, that's even worse, because it hides the page details, and displays only the domain.

When the user clicks the email link, she thinks she's opening the Apollo web site, she's right, she's on this domain. But the login form was injected, and is not coming from the Apollo server. The login form is provided by the hacker, eventually delivered to the user through an email. And then, clicking "Login" in this web page, her browser sends the login/password of her Apollo account to a remote external server, controlled by the hacker. So one gets the user password this way. This technique can be applied to Apollo's users, to get their account access. And also to Apollo employees, and their credentials could give access to internal services.

That's dreaded, because when I receive a link, one of my technique to assess for a phishing fake email, is to see the domain name of the links (without clicking them). Here, the domain name is the legitimate one, and the payload is kind of obfuscated. And today, lots of emails have very long URL for id management and click tracking. So this email can be done in a way the user is not suspicious about, and could leak further the account credentials.

My incantations to Apollo

Initially, I didn't know how I can contact the Apollo digital security department. That's an issue with many companies for digital security workers when we see a vulnerability. They don't use the RFC 9116 standard. They don't have a dedicated web page detailing any procedure to share a vulnerability. They don't share their security team email. I haven't found any bug bounty program. And their customer support was even hard to locate. Finally, I saw a generic email address, which can be used for issue reporting. So on June the 2rd, I wrote an email : "We detected a security vulnerability affecting your website www.apollo.io, which can lead to the stealing of your customers or employees credentials. Please let me know where we can reach your digital security department CISO / CSIRT / CERT, or forward this query to the relevant department." This email immediately gets an automatic reply, from a bot I think, "Thank you for contacting the Apollo Support Team. We have just received your message regarding <Security vulnerability on Apollo.io> and wanted to let you know that one of our support agents is being assigned to assist you as soon as possible." And this opened a ticket number in their issue monitoring system, number 200111591. Looks promising! Sadly, no reply from their side. I sent a remainder a week later, on June 10th : "Any feedback regarding this issue? I have no news from your side. I'm considering writing a Medium post about this and making it public.", it was registered and updated in the ticket system. Still, not a single reply to this ticket. According to the data I can see, the ticket status was changed on June 24th to "solved". The issue is still present as of today. But I have the feeling that this will be finally fixed in a couple of hours, raising the safety protection for Apollo users. [ Edit : it took approximately 15 months to be fixed, from Sept.21 when this was published and end of 2022 when fixed ]

During my research, I saw there were at least a noticeable data leak from this company. In summer 2018, their publicly gathered internal prospect database leaked 200 millions contacts in 10 million companies. They may have left a database containing billions of data points (personally identifiable) publicly exposed. That was one the biggest digital leak in history, making it in the top30. Then in March 2021, 11 millions of French workers leaks on a popular hacking forum. It is not clear if this newest datapack is a subset from the 2018 leak, or new data.

Apollo has a dedicated webpage about security. They state the security, privacy, and availability of your data is our most important priority, and we are committed to providing world-class security across Apollo's platform. This includes building a robust security system to ensure that our customers’ data remains fully secure. Our security is certified by leading 3rd-party auditors based on the most widely recognized and internationally accepted information security standards. They perform quarterly audits on access control assessment, risk assessment, information security audit, IT infrastructure audit, and HR procedures, with network penetration tests by a certified third party consultant annually. Their database is secured with multiple layers of protection, and data is encrypted at rest and in transit. They are certified ISO27001 standard compliant, and SOC-2 certified (SOC report is private). That's impressive, but personally, I often differentiate "security on paper" from "actual security". And what they provide is a lot of papers, looks like double talk.

How to solve it ? How to prevent that if I'm a digital infrastructure manager or a website coder ?

The root of this issue is a lack of sanitization of user provided data in the page URL arguments. So a simple fix is to "html escape" the data input, and it has to be displayed as html encoded text (in the inner text node). That will directly prevent the XSS. An other simple way can be to decrease the data read from the user. Because this requires a "sanitization", that's a risk, so better not to use it. In this case, don't read and display the email from url, just write "Thank you for providing your email." Is the email printing at this stage really needed? [ Edit : that was the path chosen by Apollo to finally fix this issue ]. Using a web application firewall is not designed to prevent all XSS, and don't rely on it for your security. It can only help in some cases when you make some mistakes, or run outdated software. It's an additional layer of security, for the price of breaking end-to-end encryption (yes WAF services "see" all the traffic unencrypted), you must have your core site server and content code with the less possible defects.

And there are also others measures to prevent the users from hacking, around this issue. I think about HSTS to make sure HTTPS is used, CAA to lock your certificates to a vendor, certificate transparency, and OSCP to check in real time the certificate is legitimate (Apollo is doing this). DNSSEC to prevent DNS spoofing. DMARC/SPF with DKIM to protect your email domain, preventing imposter fraud with email (impersonate "from" field). And there are specific HTML headers called Content Security Policy that are specially designed to tackle this kind of issue. Use them in a strict way can be very painful. Because the servers of a big company are not monolithic, the web content is built from various web services, with CDN, edge servers, with subdomains, with split services ranging from the technical API backend to the marketing department for tracking. At the end, using the CSP headers strictly is a good way to avoid many XSS issues. I'm not so sure CSP would help in our current specific issue, but it raises the security against XSS.

An other mean of protection, against password stealing, is second factor connection "2FA". That means that the server asks to an other secret different from the password. Using it, having only the password is not enough to connect to an account. Note that the injected HTML markup can be extended to collect a 2FA code. Using this 2FA collected code right away with the password, to login in the real website (with an automated script in the collection server), it still breaks the system. As we would have the ability to steal full users credentials, including 2FA secret, to enter in the user account.

I would also suggest to conduct regular review of the code, not only on the infrastructure, HR procedures, or network penetration, but actual and real code reading, that would have caught and fixed this issue. A simple web test drive by security specialists would have also catch this, as I did. Ultimately, I suggest to build some processes to listen to security vulnerabilities, like publishing a dedicated email "security@..", to ease the vulnerabilities report from external people. And inform the support department on how to recognize and how to handle security alerts reported to the customer issue system.

How can I protect myself, as a digital user, from this issue ?

Well, there's not so much you can do to escape this kind of phishing we imagined. It's definitively the service part to ensure strict sanitization of data inputs, and enforce many security measures to defeat XSS and phishing. But there are some behaviors you can adopt to improve your own security. The generic advice is not to rush on a email, don't click quickly to solve an emergency. Don't rush, take your time, read everything carefully, have a break. If a received email states you need to act on a short notice, it is likely a phishing attack. And take time to analyze the email data : real address of the sender, the domain of the sender. Is there a different "reply to" that has nothing related? And the links in the email, are they real, for the actual domain? Hover the link with the mouse can display the link without clicking on them. An other advanced way is to open if possible the full email as raw text, that will show you full information about the senders, how it was sent, the links,... Be very suspicious when an unknown body sends to you a web link, don't click on it. Ask your colleagues if for the work. Phone call if you can. The email "firewall" spam filtering can help a little, but they're usually not reliable, so don't rely on that.

For your web security, maintain your browser updated, don't skip updates. I recommend using Firefox. But Chrome or Edge are OK also. Using a VPN would not protect you in any way. It may just hide you if you seek for anonymity, and this relies on the VPN provider that you need to trust.

To avoid any XSS from an link by email, the best way is to go "manually" to the web site by entering its domain name, the web site address (here this means go to "www.apollo.io"), and then go with your mouse into the task you need to do what the email asked (renew, pay, disable a feature,...), using all the menus, navigating through the website. There can be alerts in your account dashboard or profile to help you about to find the task page. Of course, this can be really painful and the buttons given in the emails links are usually a way to speed up this process, as they provide direct access to the task, so we tend to use them. But sometimes the button in the email just provide a fake link for phishing, or an XSS payload as we just imagined.

Read this to go further on this "click email" subject.

Stay Safe !