Loophole in Global NCAP's protocols is affecting car crash test results

There's one specific but significant clause which I think makes no sense. Some cars can improve their safety ratings by offering worse crash protection.

BHPian ron178 recently shared this with other enthusiasts.

Those of us who've been following Global NCAP's safety ratings for a while now know they overhauled their assessment protocols in July 2022, adding more crash and safety tests than before. (Related thread)

But this thread is about one specific but significant clause in Global NCAP's new criteria that I think makes no sense. Allow me to explain why.

(There are other threads to discuss the new tests as a whole or opinions about the organisation)

Caution:
- It might be tempting to quote parts of the post out of context and jump to conclusions in the broad sense
- But this is a very specific logical flaw manifested only when specific conditions are met, not a characteristic of GNCAP or the new tests just "broadly speaking".
- Please be patient and carefully understand the problem, the number-crunching is very elementary

To keep things simple, I'll maintain this order of posts:

• Preliminaries
  Just the parts of Global NCAP's scoring criteria you'll need to know beforehand
• The problematic clause
  The paragraph that causes the problem, and Global NCAP's intention behind adding it in the first place
• Problem #1
  How a car might be unfairly penalised
• The vulnerability
  How making a car less safe can get it +1 star

Useful external links

Global NCAP 2022-25 protocols
Latin NCAP: 2016-19 protocols
Global NCAP: 2022 Maruti Suzuki Swift
Global NCAP: 2024 Citroën ë-C3
Global NCAP: 2022 Maruti Suzuki S-Presso
Latin NCAP: 2018 Renault Sandero
Latin NCAP: 2018 Nissan March

Preliminary: GNCAP star rating calculation

Before understanding what's going on, one must be familiar with how GNCAP calculates star ratings at least at the superficial level. (For a more detailed explanation, see this).

If you already know this well enough, proceed to the next post.

Test score

The maximum test score is 34 points: 16 each for the front and side barrier crash tests and up to 2 for seatbelt reminders.

Qualifiers

In addition, there are simple "yes/no" qualifying criteria for fitment or test performance of electronic stability control, pedestrian protection, seatbelt reminders, curtain airbags and the side pole impact test. But they do not add to the score.

Star rating

For each star rating, there is a minimum total score as well as requirements for each qualifier category.

A car's star rating is most often simply the highest possible rating for which it meets ALL requirements. GNCAP have nicely summed it up in this table:

Exceptional cases

Sometimes there is a special "cap" on the star rating when a body part that's important to life (head/chest) shows a very high risk of serious injury in the front or side barrier crash tests.

If numbers from the dummy show an unacceptably high risk of serious injury, the result is zero stars no matter what.

Sometimes, the dummy numbers for a body part don't necessarily show an exceptionally high risk of serious injury, but Global NCAP nevertheless thinks other factors may compromise protection. In this case, if protection for the body part is rated poor after all extra penalties, the star rating is bounded by 1 star.

The clause we need to discuss

This thread revolves around this clause added by GNCAP in 2022:

What it says, basically, is that cars whose front and side impact scores differ by more than 5.6 points (35%) will have one star chopped off their star rating.

Sounds funny already? Hold your horses. Let's understand why they're doing this.

A quick look at past protocols of Latin NCAP shows that this clause was added a few months after their first big protocol change in 2016, on which GNCAP's new protocols are actually largely based.

Trivia: SUVs and the side impact test

You see, the side impact barrier uses sits quite low to the ground and represents a small car (950kg).

This means SUVs, pickups or other vehicles with high driver seats usually do exceptionally well in the test by virtue of their height, even if they're not in the least engineered for it.

For example, this really badly engineered Great Wall pickup truck still managed nearly full points in a side impact test.

SUVs, in fact, have such an advantage in the side impact test that European legislative side impact testing straight up exempts vehicles whose driver seats are higher than 70 cm. A while ago, even the Australasian NCAP used to just give high-riding vehicles full marks and save themselves the cost of another car.

The argument for the clause

To prevent badly engineered cars from scoring high ratings by unfairly racking up points in just one test, NCAPs employ different countermeasures.

The Australasian NCAP used to require a minimum individual score in each test for a five-star rating, a move that received significant flak when reversed.

Global NCAP, instead, has introduced the 35% clause we're focussing on in this thread. To refresh your memory:

Problem #1

Let's try to understand the clause.

Remember this table?

Now look at the clause again:

Here's how I interpreted this when I first looked at the protocols:

If the difference in the front and side impact scores is >35%, then the maximum star rating that the total score can allow must be reduced by one.

That, IMO, would prevent cars from getting an undue advantage from doing well in just one test.

Global NCAP, however, doesn't think so.

To illustrate my point, I'll turn to the safety enthusiast's all-time favourite: the Maruti Swift.

Look at the table at the top of the post, and let's try to compute Swift's star rating.

Step 1: The Swift has a total crash test score of 19.19, which can allow a maximum of 3 stars

Step 2: However, the Swift has 12.9 points in the side impact, and only 6.3 in the frontal impact. That's a difference of 6.6 points (41%)!

Mm mm, the Swift is getting an unfair advantage by doing disproportionately well in the side impact, so we need to shave off a star: that makes it 2 stars.

Step 3: There are no other requirements (see table) for a 2-star rating, so that should be its final result.

Except it's not.

Clearly, there is something else making the Swift a 1-star car.

After a lot of discussion some of us had hypothesised last year that we might be interpreting the clause wrong. See discussion.

This is a little funny

Last year after scouring GNCAP's protocols for any alternate explanation, I wondered out loud if maybe - just maybe this is how GNCAP actually applied the penalty:

Step 1: The Swift has a total crash test score of 19.19 points, a maximum of 3 stars.

Step 2: The Swift didn't go through pedestrian or ESC testing (I'll ignore why, that can be a whole thread in itself), so it cannot score more than 2 stars no matter what its crash test score is.

Step 3: Because the side impact score is >35% more than the frontal impact score, deduct a star. Bingo: we finally arrive at one star.

Tell me I'm not the only one who sees the problem.

Let me break it down: Thanks to the "lack" of ESC/pedestrian protection, the Swift's good side impact score couldn't possibly have helped its star rating, which would be stuck at 2 stars no matter how well it scored. Yet the Swift was penalised, because its side impact score was...too good?

I thought I'd cope with the dissonance until there was more evidence.

GNCAP can't possibly be incompetent enough to overlook this. Right?

Proof

Thanks to Stellantis' Safety Shenanigans, all the evidence I was looking for came on a plate last week.

Look at the Citroën ë-C3's crash test result more closely:

Frontal impact: the protection offered to the driver’s and passenger’s head and neck was good. Driver’s chest showed weak protection while passenger’s chest showed poor protection limiting the result to one star. Driver’s knees showed marginal protection as they can impact with dangerous structures behind the fascia, passenger’s knees showed good protection. Driver’s tibias showed marginal and good protection and passenger’s tibias showed good protection. Footwell area was rated as unstable. The bodyshell was rated as stable and it is capable of withstanding further loadings.
Side impact: head showed marginal protection, chest showed adequate protection and abdomen and pelvis showed good protection. The difference between front and side impact made the car to lose one star in the overall result.

Let's get this straight. The ë-C3 had some 13.5ish points (out of 16) in the side impact, almost as good as it gets without side airbags.

But because of the 1-star cap because of high belt loads in the frontal impact, it wouldn't have mattered how well it scored in the side impact.

Yet GNCAP decided to penalise the ë-C3 because its side impact was so good it could be unfair (was it?).

See the problem?

The vulnerability

Unfortunately this is not just about penalising cars that are already at the bottom of the safety ladder.

Some cars can improve their star ratings by offering worse crash protection.

Outrageous claim, right? Let's turn to another Maruti, the Ignis.

It might be closely related to the Swift, but just look at the difference in side impact protection. High risks of serious skull and rib fracture, doors flying open: the Ignis' side impact protection is as bad as it can get without failing India's side impact legislation.

The Ignis is a 1-star car fair and square, because of the poorly protected chest in the side impact test.

But get this: Maruti must be thanking their lucky stars that the Ignis didn't do better in the frontal crash!

Had the Ignis scored more than 12.5 points in the frontal crash, not only would the result still be capped at 1 star because of the bad side impact, the huge difference in scores would make it a zero-star car. That is a very serious problem!

Don't believe me yet? Let's look at Global NCAP's blue-eyed boy the Maruti S-Presso.

Because of unacceptably high rib deflection in the frontal impact, it's also capped at 1 star fair and square, much like the ë-C3 is.

But thanks to its slightly worse side impact protection, its side test score is similar enough (3.77 pts or 24% difference) to its bad frontal impact, and it has escaped a 0-star rating just because its scores are consistent.

So did this Brazilian Renault Sandero that Latin NCAP tested in 2018. Bad news indeed!

For example

So, then, what's the easiest way for Stellantis to upgrade the ë-C3 to one star? (Notwithstanding the fact that they probably don't care enough to do so)

Add more safety features? Well, that could work but Stellantis has a cheaper option.

The ë-C3 has some 20 points out of which 13.5 are from the side impact. To score 1 star all that a car needs is a total of 4 points in both tests. So all the ë-C3 has to do now is escape the ridiculous penalty for inconsistency.

And the cheapest way to do that is by getting its side impact to be bad enough to match its frontal impact. Seriously, it just needs to lose some 0.6-ish points in the side test and it would score one star. It'll still be comfortably above the meagre 4.00-point (front+side) requirement for 1 star.

Surely automakers aren't petty enough to exploit a vulnerability like this, right? Of course not. They'd never do something like that.

Check out BHPian comments for more insights and information.