[ron178]

Those of us who've been following Global NCAP's safety ratings for a while now know they overhauled their assessment protocols in July 2022, adding more crash and safety tests than before. (Related thread)

But this thread is about one specific but significant clause in Global NCAP's new criteria that I think makes no sense. Allow me to explain why.

(There are other threads to discuss the new tests as a whole or opinions about the organisation)

Caution:
- It might be tempting to quote parts of the post out of context and jump to conclusions in the broad sense
- But this is a very specific logical flaw manifested only when specific conditions are met, not a characteristic of GNCAP or the new tests just "broadly speaking".
- Please be patient and carefully understand the problem, the number-crunching is very elementary

To keep things simple, I'll maintain this order of posts:

• Preliminaries
  Just the parts of Global NCAP's scoring criteria you'll need to know beforehand
• The problematic clause
  The paragraph that causes the problem, and Global NCAP's intention behind adding it in the first place
• Problem #1
  How a car might be unfairly penalised
• The vulnerability
  How making a car less safe can get it +1 star

Useful external links
Global NCAP 2022-25 protocols
Latin NCAP: 2016-19 protocols
Global NCAP: 2022 Maruti Suzuki Swift
Global NCAP: 2024 Citroën ë-C3
Global NCAP: 2022 Maruti Suzuki S-Presso
Latin NCAP: 2018 Renault Sandero
Latin NCAP: 2018 Nissan March

[ron178]

Preliminary: GNCAP star rating calculation

Before understanding what's going on, one must be familiar with how GNCAP calculates star ratings at least at the superficial level. (For a more detailed explanation, see this).

If you already know this well enough, proceed to the next post.

Test score

The maximum test score is 34 points: 16 each for the front and side barrier crash tests and up to 2 for seatbelt reminders.

Qualifiers

In addition, there are simple "yes/no" qualifying criteria for fitment or test performance of electronic stability control, pedestrian protection, seatbelt reminders, curtain airbags and the side pole impact test. But they do not add to the score.

Star rating

For each star rating, there is a minimum total score as well as requirements for each qualifier category.

A car's star rating is most often simply the highest possible rating for which it meets ALL requirements. GNCAP have nicely summed it up in this table:


Exceptional cases
Sometimes there is a special "cap" on the star rating when a body part that's important to life (head/chest) shows a very high risk of serious injury in the front or side barrier crash tests.

If numbers from the dummy show an unacceptably high risk of serious injury, the result is zero stars no matter what.

Sometimes, the dummy numbers for a body part don't necessarily show an exceptionally high risk of serious injury, but Global NCAP nevertheless thinks other factors may compromise protection. In this case if protection for the body part is rated poor after all extra penalties, the star rating is bounded by 1 star.

[ron178]

The clause we need to discuss

This thread revolves around this clause added by GNCAP in 2022:


What it says, basically, is that cars whose front and side impact scores differ by more than 5.6 points (35%) will have one star chopped off their star rating.

Sounds funny already? Hold your horses. Let's understand why they're doing this.

A quick look at past protocols of Latin NCAP shows that this clause was added a few months after their first big protocol change in 2016, on which GNCAP's new protocols are actually largely based.

Trivia: SUVs and the side impact test

You see, the side impact barrier uses sits quite low to the ground and represents a small car (950kg).

This means SUVs, pickups or other vehicles with high driver seats usually do exceptionally well in the test by virtue of their height, even if they're not in the least engineered for it.

For example, this really badly engineered Great Wall pickup truck still managed nearly full points in a side impact test.

SUVs, in fact, have such an advantage in the side impact test that European legislative side impact testing straight up exempts vehicles whose driver seats are higher than 70 cm. A while ago, even the Australasian NCAP used to just give high-riding vehicles full marks and save themselves the cost of another car.

The argument for the clause

To avoid badly engineered cars from scoring high ratings by unfairly racking up points in just one test, NCAPs employ different countermeasures.

The Australasian NCAP used to require a minimum individual score in each test for a five-star rating, a move that received significant flak when reversed.

Global NCAP, instead, has introduced the 35% clause we're focussing on in this thread. To refresh your memory:

[ron178]

Problem #1

Let's try to understand the clause.

Remember this table?


Now look at the clause again:


Here's how I interpreted this when I first looked at the protocols:
If the difference in the front and side impact scores is >35%, then the maximum star rating that the total score can allow must be reduced by one.

That, IMO, would prevent cars from getting an undue advantage from doing well in just one test.

Global NCAP, however, doesn't think so.

To illustrate my point, I'll turn to safety enthusiast's all-time favourite: the Maruti Swift .




Look at the table at the top of the post, and let's try to compute the Swift's star rating.

Step 1: The Swift has a total crash test score of 19.19, which can allow a maximum of 3 stars

Step 2: However, the Swift has 12.9 points in the side impact, and only 6.3 in the frontal impact. That's a difference of 6.6 points (41%)!

Mm mm, the Swift is getting an unfair advantage by doing disproportionately well in the side impact, so we need to shave off a star: that makes it 2 stars.

Step 3: There are no other requirements (see table) for a 2-star rating, so that should be its final result.

Except it's not.

Clearly, there is something else making the Swift a 1-star car.

After a lot of discussion some of us had hypothesised last year that we might be interpreting the clause wrong. See discussion. (Global NCAP safety ratings shouldn’t be reduced to Sir Boastalot)

This is a little funny

Last year after scouring GNCAP's protocols for any alternate explanation, I wondered out loud if maybe - just maybe this is how GNCAP actually applied the penalty:

Step 1: The Swift has a total crash test score of 19.19 points, a maximum of 3 stars.

Step 2: The Swift didn't go through pedestrian or ESC testing (I'll ignore why, that can be a whole thread in itself), so it cannot score more than 2 stars no matter what its crash test score is.

Step 3: Because the side impact score is >35% more than the frontal impact score, deduct a star. Bingo: we finally arrive at one star.

Tell me I'm not the only one who sees the problem.

Let me break it down: Thanks to the "lack" of ESC/pedestrian protection, the Swift's good side impact score couldn't possibly have helped its star rating, which would be stuck at 2 stars no matter how well it scored. Yet the Swift was penalised, because its side impact score was...too good?

I thought I'd cope with the dissonance until there was more evidence.
GNCAP can't possibly be incompetent enough to overlook this. Right?

Proof

Thanks to Stellantis' Safety Shenanigans, all the evidence I was looking for came on a plate last week.



Look at the Citroën ë-C3's crash test result more closely:

Frontal impact: the protection offered to the driver’s and passenger’s head and neck was good. Driver’s chest showed weak protection while passenger’s chest showed poor protection limiting the result to one star. Driver’s knees showed marginal protection as they can impact with dangerous structures behind the fascia, passenger’s knees showed good protection. Driver’s tibias showed marginal and good protection and passenger’s tibias showed good protection. Footwell area was rated as unstable. The bodyshell was rated as stable and it is capable of withstanding further loadings.
Side impact: head showed marginal protection, chest showed adequate protection and abdomen and pelvis showed good protection. The difference between front and side impact made the car to lose one star in the overall result.

Let's get this straight. The ë-C3 had some 13.5ish points (out of 16) in the side impact, almost as good as it gets without side airbags.

But because of the 1-star cap because of high belt loads in the frontal impact, it wouldn't have mattered how well it scored in the side impact.

Yet GNCAP decided to penalise the ë-C3 because its side impact was so good it could be unfair (was it?).

See the problem?

[ron178]

The vulnerability

Unfortunately this is not just about penalising cars that are already at the bottom of the safety ladder.

Some cars can improve their star ratings by offering worse crash protection.

Outrageous claim, right? Let's turn to another Maruti, the Ignis.

It might be closely related to the Swift, but just look at the difference in side impact protection. High risks of serious skull and rib fracture, doors flying open: the Ignis' side impact protection is as bad as it can get without failing India's side impact legislation.

The Ignis is a 1-star car fair and square, because of the poorly protected chest in the side impact test.



But get this: Maruti must be thanking their lucky stars that the Ignis didn't do better in the frontal crash!

Had the Ignis scored more than 12.5 points in the frontal crash, not only would the result still be capped at 1 star because of the bad side impact, the huge difference in scores would make it a zero-star car. That is a very serious problem!

Don't believe me yet? Let's look at Global NCAP's blue-eyed boy the Maruti S-Presso.



Because of unacceptably high rib deflection in the frontal impact, it's also capped at 1 star fair and square, much like the ë-C3 is.

But thanks to its slightly worse side impact protection, its side test score is similar enough (3.77 pts or 24% difference) to its bad frontal impact, and it has escaped a 0-star rating just because its scores are consistent.

So did this Brazilian Renault Sandero that Latin NCAP tested in 2018. Bad news indeed!

For example

So, then, what's the easiest way for Stellantis to upgrade the ë-C3 to one star? (Notwithstanding the fact that they probably don't care enough to do so)

Add more safety features? Well, that could work but Stellantis has a cheaper option.

The ë-C3 has some 20 points out of which 13.5 are from the side impact. To score 1 star all that a car needs is a total of 4 points in both tests. So all the ë-C3 has to do now is escape the ridiculous penalty for inconsistency.

And the cheapest way to do that is by getting its side impact to be bad enough to match its frontal impact. Seriously, it just needs to lose some 0.6-ish points in the side test and it would score one star. It'll still be comfortably above the meagre 4.00-point (front+side) requirement for 1 star.

Surely automakers aren't petty enough to exploit a vulnerability like this, right? Of course not. They'd never do something like that.

[Aditya]

Thread moved out from the Assembly Line. Thanks for sharing!

[IshaanIan]

I have been saying for several years that people should not simply look at a star rating and that they really ought to read the finer details before drawing any conclusions with these ncap star ratings. This is really bad though however will this only affect cars that are rated poorly (3stars and under) or can there even be a scenario where this can affect a 4 or 5 star car because if that happens then we could expect manufacturers to take the easy way out and make compromises just to secure 4 or 5 stars.

[MotoBlip]

Thanks for providing this insightful point of view. I have always found NCAP ratings a bit funny and have criticized them in my posts for their lopsided approach previously, although they fixed some of these issues with their next iterations. Also, there were points regarding transparency and vendor lobbying raised by a big YouTuber regarding their results, but I believe that's best taken with a pinch of salt. I don't think this is going to affect the highly rated cars (4/5 stars); historically, it's not like these specific cars were over-promising and under-delivering. However, as IshaanIan mentioned, it's important to read the fine print, especially with lower-rated cars.

[sridhar-v]

Thanks a zillion for this insight.
I would always prefer to go by the actual damage colours on the dummies.
And how the structure behaves in the actual crash tests.

[tbppjpr]

Problem is when the crash test ratings also turns out like any other marketing gimmick for the manufacturers. Most people don't go through the analysis data, they just count the stars and thats it.

Cars like Verna with unstable body-shell getting 5 star ratings raise more questions on their credibility.

[antz.bin]

This loophole is relevant only to the laggards who missed adding essential equipment such as seatbelt pre-tensioners and ESC. They are anyways not worth any safety credentials. Does it really matter if something gets a zero or a 2 star rating? Both are bad when a similarly priced 3/4 star car is also available in the market.

The point here is, if someone is wilfully opting for a low rated car, the person is unlikely to read the fine print when he/she didn't even bother with the large letters and pictures of stars with colours filled and stars with missing colours. That person will probably hand a nearby toddler a yellow crayon to fill in the unfilled stars.

I agree that old test process scores cannot be directly compared with new ones and a new 1 star car could potentially be safer than an old test process 4-Star rated car. But where I come from, 2/5 (40%) and below is a fail, regardless of what test process was used. (We had to score at least 41% to pass our college exams .). This was not a surprise test. The test process is being used since 2022. The entirety of 2023 has passed in between.

Also, instead of reducing performance in the side impact test, there is an even easier process to improve scores. Just add the bloody pre-tensioners and ESC and have the capping from the scores removed. You could very well score 3-Stars with that instead of score-hacking towards another fail score. Improve the frontal impact scores and you are looking at 4-Stars instead of a zero. At least that way the e-C3 could show the world that their safety rating is objectively better than Swift and Ignis.

[07CR]

Extremely well put. I never gave much importance to GNCAP tests, since they have had so many loopholes. Thanks for pointing one more glaring one.

I just wish BNCAP improves on the GNCAPs loopholes.

[ron178]

Originally Posted by IshaanIan

will this only affect cars that are rated poorly (3stars and under) or can there even be a scenario where this can affect a 4 or 5 star car

An excellent question. Let’s keep in mind which consequence we’re talking about.

#1: a car being penalised for claiming an advantage it didn’t get
If there is a reason for a car not to get an advantage from a better score, that probably means it either has very poor protection in a test (eg. eC3) limiting it to 1 star, or it doesn’t have safety tech like ESC or curtain airbags. Then it’s easy to look at the big picture and say: these cars are already show very little ambition towards state-of-the-art safety, does +/- 1 star, even if unfair, really matter?

#2: encouraging reduction in score
This one actually can affect all kinds of cars.

We’ll consider a car that does have all the requisite features to meet the additional requirements for ESC, seatbelt reminders, pedestrian protection, side pole test etc.

To understand it better, let’s plot a simple graph with the front and side impact scores on the horizontal and vertical axes.

We’re trying to identify a situation where a car has a total score for 5 stars, but is being affected by the huge difference and winds up with 4.

For this, all of the following would be satisfied:
front + side >= 25, to score enough total points for five stars (excluding 2 for seatbelt reminders)
| front - side| >= 5.6, to be penalised for variation in front and side impact scores
front <= 16
side <= 16, the maximum available points in each test

The intersection of these is two small triangles showing cars that could have been five stars, but aren’t, because of the penalty.


The triangle highlighted in blue represents cars that are penalised for too good a side impact score.
The one highlighted in pink represents those penalised for too good a frontal impact score.

Consider a car in the blue-highlighted triangle: say 10 points in the frontal impact and 15.6 in the side impact (the black X in the figure below).



To upgrade to 5 stars, the vehicle manufacturer needs to move the score to a point in the red area, but outside the triangle.

A good manufacturer would follow the green arrow: improve the frontal impact score.

An irresponsible manufacturer could just follow the red arrow: reduce the side impact score.

As the target star rating reduces, the size of the triangle increases, as does the probability of winding up in this situation.

Ideally, the right boundary should have been a vertical (not sloping) line, just as the upper boundary of the lower triangle should have been horizontal. Just like ANCAP when they used to require a hard minimum 12.5 on each test to score 5 stars.

It doesn’t mean Global NCAP are intentionally favouring cars out of malicious intent. They just haven’t thought this particular clause through at all.

[r2212xx]

It is not just this loophole but various loopholes or grey areas, including - Percentage difference calculation, sometimes I get the feeling, for some cases, they used the Percentage change formula rather than Percentage difference, What defines “Exceptional cases”, such as Wagonr scoring poorer with one star than Alto K10’s two stars despite both having similar results, What impact does the front co-passenger’s rating have since I have a suspicion there is some undisclosed rating associated with it, and, of course, How do cars with “unstable bodyshell integrity” still manage to score five stars.

The more one reads their reports, the more questions one has. I do not think that GNCAP is unaware of this or has some arbitrary factors; they just find it, for some reason, difficult to put it down as simple lettering. In our country, where there is a lot of "perceived safety" associated with brands and people can remove or add their own stars during watercooler chats, further elaboration of the rating nuances is warranted, considering GNCAP ratings have become the new marketing tool for automakers.

[IshaanIan]

Having read the details on many car safety tests, I much prefer the North American IIHS standard of testing. I wish we could add something like that here in our country we can even copy the entire thing and just call it the Indian Institute of Highway Safety and it would also be more comprehensive by covering under-run bar protection on trucks as well as perhaps poke the NHAI to incorporate better guard rails and lead to a more comprehensive safety outlook.