Codeship Security Vulnerability Disclosure

The Codeship service experienced a security vulnerability in our authentication service on March 10 and 15, 2015 that impacted four users and allowed unauthorized access to account data. This issue was resolved on March 15, 2015 with an intermediate fix and permanently resolved on March 22, 2015.

Codeship takes the security of our systems and protection of our customers’ private data very seriously. An important aspect of maintaining the trust of our customers in regard to security is to have complete transparency, disclose security issues when they arise, and discuss our response.

Two separate but related incidents occurred on March 10 and 15, 2015 that compromised the account/project data for two separate Codeship users. Their code was not impacted, but sensitive information stored with their project configuration was revealed. We are very sorry this occurred and are taking the opportunity to examine every aspect of our handling of the incidents, and how we approach security, in general.

Incident Details and Response

On March 10, 2015 we were informed by a user that they had gained access to another Codeship users’ account without taking any action to attempt to do so. They simply navigated to their projects page and became authenticated as another user. They immediately reported the issue to Codeship Support where it was escalated and investigated by the Engineering team. Evidence of the issue was found in our logs and analysis showed that it was an isolated issue. Detailed review of our code allowed us to determine that a vulnerability existed in our use of an open source authentication library for Ruby. Rare circumstances could allow for the value of a secure token stored in a cookie to collide with another user's’ token. We shipped code to close this vulnerability on March 11, 2015 that included switching our session store from encrypted cookies to shared memory store on the server.

On March 15, 2015 we received another report with the exact same description as the earlier report. Log analysis again showed that the issue was isolated, and investigation showed that our attempt to fix the vulnerability, shipped on March 11, 2015 was not successful due to incomplete understanding of the problem. An intermediate fix was applied on March 15, 2015 to definitively close the vulnerability while a larger effort was started to completely remove the use of the open source authentication library from our source code. The intermediate fix did result in removing the “Remember me’ authentication functionality. On March 22, 2015, an update to Codeship was released with a completely new authentication scheme and the third party library removed.

At this time, we have not definitively identified a security vulnerability in the library (the problem may have arisen because of our implementation), however we remain committed to working with the project maintainers to identify and fix should our investigation find that it is necessary. The decision to replace our use of the third-party library was not taken lightly as it is a well known and maintained project, however our use case only utilized a small percentage of the overall codebase of the library, and we were interested in simplifying our authentication implementation for long term maintainability.

Future Improvements

After completing a post-mortem review of the handling of these issues, some areas for improvement were identified, and we are undertaking a serious effort to address them. Communication is the first area that was identified. We are committed to doing a better job communicating with our customers about security vulnerabilities, and we will be launching a formal security response program providing a venue for the responsible disclosure of issues and the ability for us to work directly with independent security researchers.

We hope that with the help of independent security researchers along with security professionals directly contracted by Codeship, periodic assessments and audits can help us find and prevent security vulnerabilities in our application.

We would like to offer our sincere thanks to the parties involved with the incidents outlined above for their responsible disclosure, handling, and most importantly, for their understanding.