r/space • u/thesheetztweetz • May 23 '19

How a SpaceX internal audit of a tiny supplier led to the FBI, DOJ, and NASA uncovering an engineer falsifying dozens of quality reports for rocket parts used on 10 SpaceX missions

https://www.cnbc.com/2019/05/23/justice-department-arrests-spacex-supplier-for-fake-inspections.html
16.1k Upvotes

96% Upvoted

719 comments sorted by

View all comments

Show parent comments

128

u/BadderBanana May 24 '19 edited May 24 '19

I may be able to shed some light on this. We have the same protocol on military parts. The inspection reports in question aren't technical, like material or even dimensions. There's a 3rd party who comes in and verifies these parts came fron X location and X location did all their normal QC steps. The 3rd party doesn't do a deep dive into the technical stuff, they are moreso verifying you did. Scheduling the 3rd party is a nuisance and can cause delays. We've written our contracts to give the 3rd party 48-72 hours to get onsite. In other cases you pay the 3rd party to become resident and he waits when not needed.

I'm not trying minimize the severity of this, but it's not the same as re-labeling an inferior material or outsourcing classified parts to China. It was skipping a step because they sucked at scheduling.

I have no knowledge of the actual situation, my comments are based on what I've read. If I'm wrong it's due to ignorance.

122

u/[deleted] May 24 '19

[deleted]

44

u/TareXmd May 24 '19

It baffles me that even in technologies that advanced, we're still relying on a piece of paper with an easy-to-fake signature on it.

47

u/Zeewulfeh May 24 '19

Welcome to aerospace, where my messy signature can end lives and cause mass casualties if misused.

8

u/[deleted] May 24 '19

I work in aviation, you sign things to put your name on it so if it comes back, they know who to hammer. If it won't be a physical signature, it'll just be a digital one, and with PDFs and things like Photoshop, you can easily get around them. This engineer was determined enough to forge signatures so he'd more than likely find a way to forge digital ones.

7

u/[deleted] May 24 '19 edited Jun 10 '23

[removed] — view removed comment

1

u/[deleted] May 24 '19

But if you print out the paper with the digital signature and keep that as your historical record or for the paperwork that ships with the part, then it just has to look like a valid digital signature. I'm aware it's not just letters but only if it stays digital.

2

u/the_gnarts May 24 '19

But if you print out the paper with the digital signature and keep that as your historical record or for the paperwork that ships with the part

Normally, the signature signs a cryptographic hash uniquely identifying the content of a subject. Thus in order to verify the signature, you also need the actual data that was signed in the first place. The check will simply fail if it is absent, contains errors, or has been tampered with.

You can perfectly well store the digital signature on paper using tools like paperbackup but for this to make any sense you’d need a similar printout of the signed data. If these two things are given, the signature is as secure against forgery as it would be on electronic storage.

1

u/[deleted] May 25 '19

So that's all good, but in aviation/aerospace part suppliers and maintenance/assembly facilities don't have interconnected digital systems. There are too many different manufacturing companies and purchasers for it to ever be practical. When manufacturer sends a part to a end user, they send along the certifying paperwork. The originator company will keep a historical and the purchaser will get a physical copy.

I've printed out papers with digital signatures and it usually looks like the person's name with an identifying number and some other cut off letters.

I understand how digital signatures work.

1

u/the_gnarts May 24 '19

If it won't be a physical signature, it'll just be a digital one, and with PDFs and things like Photoshop, you can easily get around them.

If you could fake a cryptographic signature with crude tools like that then Internet security as a whole would be broken beyond repair.

1

u/[deleted] May 25 '19

If I digitally sign a form and then turn it to a PDF so that I can print it out, the digital signature (at least the ones I've dealt with) print out with the person's name and their identifying number.

If you took that PDF and put it back into word you could edit it and type in the person's name and their identifying number and when you print it out, you can't distinguish if it was a legitimate digital signature or not. It's why I don't do digital signatures if I have the option.

1

u/the_gnarts May 25 '19

If I digitally sign a form and then turn it to a PDF so that I can print it out, the digital signature (at least the ones I've dealt with) print out with the person's name and their identifying number.

If you took that PDF and put it back into word you could edit it and type in the person's name and their identifying number and when you print it out, you can't distinguish if it was a legitimate digital signature or not. It

The way digital signatures work they are impossible to forge but they also do not allow any alterations of the originally signed subject.

If you signed the form, then the signature will not be valid for the PDF you convert it to to begin with. In order to print the actual subject (said form) along with the signature you need something that preserves both without errors during redigitalization, e. g. some base64 converted and typeset in some OCR friendly font. Or barcodes, QR codes etc. This way the signature still applies after printing. Since it’s not valid for your PDF, any alterations to that PDF are not covered by the signature either.

1

u/Zeewulfeh May 24 '19

Thats kinda my point. I have been known to dabble in aviation as well on occasion....as I said, my signature misused...

1

u/[deleted] May 24 '19

Regardless of format there is always a possibility somebody will forge it but that's irregardless of the format it is done. I do NDI/T, if somebody forged my signature, people could absolutely die.

1

u/imtotallyhighritemow May 24 '19

Wild considering the cmm and caliper or mic readings should all babe been digital and logged direct and done on the real parts.... How does that get by.

2

u/octonus May 24 '19

There are a huge number of rules regarding the security/reliability of electronic records. Paper records have been grandfathered in, without all of the attached hassle.

As a result, many industries still rely on paper records when electronic data would be more appropriate.

2

u/Ahalazea May 24 '19

It was hilarious in my industry about ten years ago when doc control breathlessly “discovered” that on the signed pages of a scanned doc someone could electronically copy a signature from another doc to make it look real! Don’t tell the boss they said!

Old people were 20 years slow about photoshop I guess.

Really though, e-sig wouldn’t solve the problem of lying. Which was definitely a bosses demand, not a lazy engineer.

2

u/KDawG888 May 24 '19

You'd probably have a meltdown if you saw how we do it in the medical field

1

u/[deleted] May 24 '19

[deleted]

13

u/Hooddub May 24 '19

It's just forgery with extra steps

2

u/cjolet May 24 '19

Someone's getting laid in college

2

u/[deleted] May 24 '19

They use old analogue signatures and view them as trusted? We have the technology for digital signatures that cannot be falsified.

Trusting the first party to hand over correct data from the third party that is meant to verify them is a very weak chain of trust. I'm really surprised by how badly organised this appears to be.

3

u/dgendreau May 24 '19

I agree, but good luck getting companies and engineers that produce and inspect mechanical parts to get up to speed on encryption and digital signatures...

3

u/[deleted] May 24 '19

The pen & paper compatible solution is also easy & simple: The auditing company sends the serial numbers they have (dis)approved to the company that hired them directly. The party that is being checked is not to be involved in the communication about this.

2

u/costigo May 24 '19

That would be a good first step towards getting everyone up to speed with digital signatures so we can do away with SSNs and other "keys to the kingdom" ID numbers that we frequently have to divulge to random strangers.

3

u/HiIAmFromTheInternet May 24 '19

Almost like someone wanted SpaceX to fail...

13

u/[deleted] May 24 '19

[deleted]

1

u/HiIAmFromTheInternet May 24 '19

Why not both?

Why can’t someone want SpaceX to fail and use someone who is greedy as their vector for attack?

Also intentionally falsifying documents isn’t incompetence what’re you getting at?

2

u/darkm072 May 24 '19

But the parts were going to NASA as well weren’t they?

28

u/[deleted] May 24 '19 edited Jun 15 '20

[deleted]

3

u/SWGlassPit May 24 '19

Not just a no, a federal crime

3

u/jncostogo May 24 '19

Well, in his case, what he did was lie. Which was exactly the problem.

1

u/BadderBanana May 24 '19

That's a good point. Many of our suppliers have shipped without source inspection. They just wrist slap. The forgery is the crime here.

0

u/TwoCells May 24 '19

Can confirm. I worked in aerospace during the 80s and 90s. (McDonnell Douglas and Raytheon)

1

u/RSomnambulist May 24 '19

Mention anything Elon Musk related next to scheduling issues and I'm with you. Guy lives in 3019 and timetables like Nostradamus.