PCI: Not our problem...

Posted by Tate Hansen Thu, 17 May 2007 02:51:00 GMT

What happens when the test environment operated by MasterCard (they “own” the testing lab) is misbehaving? I know. They yank the wheel, swerve away from responsibility, and point to the PCI council. And PCI? They point back. Beautiful, no?

You see because they refuse to disclose missed results to you they duck responsibility for anything that may have been their fault. They also clearly imply if anything is missed in your attempts to identify vulnerabilities then it is surely your fault or a problem with the tools you used.

I love it: No clear pass criteria, no way to challenge a decision, and no transparency of what or how they are doing. For all this great service you get to spend thousands every year!

So what happens when you call bullshit and raise hell? They pass you. :) Let me not forget to mention we had a few extra bullets in our clip they may have unexpected us to have – bullets provided to us by friends with information.

Be forewarned; this process has serious issues.

Tags , , , , , , , , , , ,  | 1 comment

PCI ASV wonderland

Posted by Tate Hansen Fri, 04 May 2007 17:52:00 GMT

The world of the PCI Security Standards, ASVs (Approved Scanning Vendors), and commercial scan vendors is, from my limited interactions, not exactly on the straight.

Having recently spent considerable time preparing, scanning, and writing reports with the explicit goal of becoming an ASV, I’m disturbed by my communications with all involved.

It doesn’t help that the pass criteria to become an ASV is not clear. Is it based on discovering all vulnerabilities on their test network? A subset? Which parts are subjectively reviewed?

I like to use Qualys to baseline vulnerabilities, which a test representative caught as one of the tools we’d be using based on the source IP blocks for scanning. He said something like “If you use Qualys, you’ll get 95% of what you need”. From that I guessed the example web application would have vulnerabilities which would be missed by Qualys and other network-based tools. As expected, that was true.

Fast forward. We received feedback that our reports have not been reviewed because Qualys changed something recently causing expected results to be absent which PCI requires for passing. The representative said “If you’d have scanned a month ago with Qualys, you’d have passed with flying colors”. He added that they are in communication with Qualys to resolve the issue.

Nevermind that Qualys was only one of the tools we used and we had added vulnerabilities not discovered by the popular free or commercial scanners. It was clear the representative didn’t review the report, which he said as well (and may have done to protect us from an automatic failure -- even though our current pass status is pending). But they would not reveal the gaps, which obviously makes it hard to understand what the problem is. I appealed by mentioning I had added vulnerabilities not discovered by Qualys. He then modified his previous statements by saying he in fact spot checked the reports and the items he was looking for were absent. My gut reaction to all this: bullshit.

The connections apparent make for a nice racket. You pay lots of money to PCI. PCI “communicates” with selective vendors to ensure all the vulnerabilities they expect to be discovered are discovered. You pay money to the scan vendors. And what about if you find additional vulnerabilities or a superset? Sounds like they don’t check and don’t care. I would think the idea behind all this is to make sure you are adding sufficient value to entities subject to PCI regulations (even if that means you didn't catch 100% of everything bad). If passing simply means doing a blind Qualys scan when it works right (i.e. does what PCI wants), well then, you now know what to do to as an attacker -- just go after something Qualys doesn't check.

So much for trusting this process and what it does to vet competent assessment companies.

Tags , , , , , ,  | 1 comment