Update: New Calibration information, issues presented at bottom is an anomaly of calibrating off the bike, see here.
In response to questions on accuracy of my V3 Prototype that was shown here and featured on Hackaday I’ve been doing some testing. The most recent testing is based on comparison to the Powertap G3. This is going to be broken up into a few steps
- The Software
- ANT+ Communications
- Test 1
- Correction
- Test 2
- Calibration
- Lessons Learned
The Software
The above software looks ultimately dull, and I’ll admit it is. However it’s functional (though can crash if I do something stupid I suspect) and does something out there no software does. It records all data fields transmitted by a power meter (except crank frequency, but nothing uses that data page that I know of) and it does it every 0.250 second or 4hz. I’ll get to why that is important in the ANT+ communications bit, but suffice to say that the data from 2 powermeters is recorded and synchronized. I rented the G3 on a Tuesday – I didn’t get the software written until Friday, so 2 days of missed testing.
ANT+ Communications
From top to bottom you’ll see B Watt, C Watt, W watt, C N/M, C RPM, W N/M and W RPM. This will become clear soon but suffice to say B, C, W are for Basic, Crank, and Wheel.
In the power profile for ANT+ it defines pages. There is an 8 byte message to every payload that happens every 0.25 seconds, and the first byte defines the page. Sadly this is a bit of a waste of data but necessary. A basic powermeter transmits only the basic page. This is the bare minimum for compatibility and if your receiver supports the basic profile it can communicate with ALL powermeters. That’s correct. You might be wondering then what about the crank and wheel profiles? Every 4 messages these powermeters have to send a basic page!
So a crank will go:
CCCCBCCCCBCCCCBCCCCBMCCCBCCCB
If you bother to read all that you’ll see an ‘M’. That M is the manufacturers information containing serial numbers, manufacturer ID etc. These come sporadically working out to every few seconds.You’ll notice then that a head unit has two means of displaying power. In practice I’ve found that the Garmin Edge 500 I have is pretty clever in what is going on in the background. If you start it up and the powermeter starts sending basic pages it’ll start extracting power data from that, but if you switch to crank or wheel it ignores these pages. However if the computer starts up and sees crank or wheel data it’ll calculate the power from the torque and rotational velocities presented, but if you switch to only basic power transmission then it doesn’t show power – you have to reboot it or research for the powermeter. If your head unit is only seeing basic profile for a crank or wheel unit then your 1hz recording is now oversampling because you only get a basic page every 1.25 seconds.
What I’ve presented, changing profile types, would not be approved by the ANT+ committed, but it’s worth noting that these devices expect consistency and what page they get their data from can change how fast you get your data.
I noticed the Powertap updates it’s wheel page only when the basic page is updated. GASP! That means you are only ever getting data at 0.8 hz from a Powertap G3!
However, let us assume that a wheel based meter or crank based meter updated every rotation, and at high speeds at around 200+ rpm that means almost every page it sends is different. You get 4 times the data you get with the basic profile potentially. If the basic page comes around it syncs well with the wheel page but it isn’t averaged for the last 4 pages you’d notice error. So if you’re wheel is spinning 240 rpm or about 30.2km/hr, your data could have error – with itself! Using the ANT+ Sensor Simulator and my software I can show you this error below. Notice how the blue basic page lags the more accurate red crank page.
Below is some real Powertap G3 Data. No real error, however this means that the G3 could actually have greater accuracy potentially and may actually answer why it sometimes doesn’t line up perfectly with the crank based meters. Honestly, I think that Cycleops should have the Powertap updating every revolution, but I’m more engineer than Coach – to a coach, it doesn’t likely matter.
Test 1
Utilizing both the Powertap G3 and my V3 Prototype produced the graph below. Both meters follow similar trends very closely. Initially I had to restart my prototype to re-zero it a couple of times. Once that is complete and the V3 is responding well it follows much better, though offset lower.
Below is a blow up of my meter versus the Powertap G3 to highlight the differences. One of the things you’ll notice is the higher response of my meter versus the G3. This again goes back to how the processing happens. In theory the wheel is spinning faster so should update twice as often as the crank but it does not.
The difference between both meters hovers around zero, however there is some offset on average. There is some larger errors coming up during the intervals.
Mapping my meter against the G3 shows good linearity, but showing a slightly greater slope. When I calculate out the average torque and gear down during intervals it appears there needs to be an 11% torque correction. The bits/torque is therefore decreased 11%.
Correction
Increasing the torque by 11% from the first test yielded the results below. The data now lines up very well.
Again, plotting my meter versus the G3 shows a better relationship, though potentially overcorrected. This is correct as we’ll see in the next test.
Test 2
Using the data from the Powertap and the V3 Prototype with the correction factors yielded the following test. Once corrected they give good alignment with the G3. The blue indicates my powermeter while the red indicates the Powertap G3. One thing to note that was mentioned earlier – the Powertap G3 is updating the Wheel page ONLY after it updates the basic page whereas I am only using the basic page. The Wheel is usually spinning much faster than the crank, say approximately 2:1. But the G3 only updates once every 1.25 seconds, so I’m really achieving 2:1 more data approximately with my prototype as was shown previously. Near the end of the last interval I wanted to push it and you can see the response of my powermeter leads that of the Powertap clearly showing the advantage of more updates.
Once I implement a 1 second smoothing algorithm (that would likely be used in most cycle computers) this becomes the graph below. As you can see, it’s lining up even better.
Similar to the graphs shown before, the comparison of each meter by graphing the power of the V3 Prototype against that of the Powertap we get the following. There is certainly scatter as the Powertap can be delayed in time up to a second causing offset forward or backward. The regression now indicates that the new calibrations are over estimating power compared to that of the Powertap. This is more in line with what should be expected as there is a certain amount of drive train loses to be compensated for. It is very close to what was expected based on the 11% change in terms of linear regression.
The average offset works out to be about 7 watts. I can again drive this back down. The minor slope indicates good correlation and little drift resulting from the room warming up. There is still a little. The new ADS1247 has a thermal sensor built in, and the nRF51422 also has a thermal sensor built in. I can therefore use these to do any thermal compensation needed.
Calibration
Above is a picture of a $32.99 dollar strain gauge based weight scale that measures up to 5kg in 1g increments. I tested 15kg worth of weight and recalibrated. What I found was that my right crank arm calibration was somewhere between the original and the Powertap G3 Adjusted and my Left one ranged. Wait, what? How can that be.
Apparently my alignment of the strain gauge in the axel is not perfect and as a result the hard the effort on the left leg the greater the bending moment combined with the twisting moment and causes a nonlinear calibration. It’s not bad, but it’s certainly an issue. There was a bare hint of this on the right side, but not much.
Lessons Learned
Strain gauge alignment. This is more critical than expected. At work I’ve made alignment gauges, here I assumed that the self compensating nature would take care of most of it, and to an extent it does, however 2 degrees of angle better and it might never have been an issue. The right measurement using the double bending bridge arrangement is good but the left will need serious attention to alignment and installation.
Also, notice I put in the price of a strain gauge scale. If there was an easy way to combine this with a 50 dollar Garmin speed / cadence sensor you would have a powermeter essentially. I mean the scale has a microcontroller in it already and a display and fancy stainless steel flat. Powermeters aren’t complicated in terms of engineering, but they have their issues to build.