Improving Quality

As you might have noticed I now have merged Gavin’s MSF code into my development branch at Github. So far I did not put it into the master branch as part of a release. The point is that I still did not test it. The same holds true for the contributions of Aido.

Before I merge this stuff into master I want to be sure it works. Hence I ask everyone who wants to contribute please give me some feedback to one or both of the following questions:

  1. Does the MSF Code work for you?
  2. Does the (DCF77) Code work with an Arduino Leonardo?

I also improved the documentation quite a lot.

But the most work went into setting up unit tests for my library. As it turns out I introduced some subtle issues into my code while refactoring. Now I uncover and fix them one by one. Continue reading here to learn about the details.

This entry was posted in Uncategorized. Bookmark the permalink.

20 Responses to Improving Quality

  1. paxyinrs says:

    Keep a great job !
    Project works great with Crystal Mini Pro and do not work because of drift at Ceramic Nano.
    I have just combined Swiss Army Debug Helper with MB Emulator so I can with option “Dn” switch to NTP mode (MB Emulator). That way, with Scope and Debug I first find best position, with lowest noise level then I switch to MB emulation. In future, I will probably add some hardware switch for moving form one mode to another.

  2. Emmanouil A. Neonakis says:

    Great Work.
    I am thinking of using the once per second output as a time base for measuring network frequency.
    Is it in your opinion accurate and stable enough for such a use? In particular how much is the period variation?
    Thank you.

    • In principle yes. The details are somewhat involved though. Once the signal is locked to DCF77 it will stay “exactly” in sync with DCF77. However there is a quantization error of +/- 1ms. Also you have a crystal frequency deviation. Lets say for the sake of example your crystal is running at 16 MHz -50 ppm. Then your second ticks will be ~ 1.00005 s appart. Also every 20 seconds you will get a second tick that is only 0.99895 seconds long (equivalent to a phase glitch of pi/20 with regard to mains frequency). Once the clock is running for a several days it will autotune to this deviation and persist this to flash memory. Afterwards all manufacturing related deviations will be gone. Thus the crystal deviation will be soleley due to againg and temperature deviations. If your are monitoring mains frequency the device will be probably running all the time. Thus due to automatic retuning you will be left with the deviations due to temperature. Or with other words maybe +/-2 ppm. Thus you will get the 1ms jump every 100 seconds. It will nevery be more than 2 ms out of phase. That is the deviations do NOT sum they will always even out to keep in phase with DCF77.

      Since it phase locks to DCF77 (like swissgrid does) this should be sufficient if you know what you are doing.

      With other words: if your algorithms account for this behaviour then it is OK, otherwise not. If you use it for a project I would be happy to learn about the details 🙂 In particular I like to learn about the results.

      • Emmanouil A. Neonakis says:

        Thank you.
        The quantization error of +/- 1ms applies to Arduino Uno or to Due?
        Does the tuning algorithm eliminate the quantization error or only the crystal deviation?
        I will certainly inform you about the details if I finally proceed with this project.
        But, as I lack the necessary instruments, I will not be able to assess the accuracy of the measurement.

      • It can only compensate crystal deviation. The +/- 1ms applies to the Due. For the “Uno” it is +/- 10 ms. Please notice that the Uno is NOT crystal based. You will have to find a clone that actually features a crystal.

  3. Emmanuel A. Neonakis says:

    That unfortunately rules out the UNO. And it is not realistic to replace the DUE crystal with a TCXO.
    By the way do you know if the library has been used on teensy 3.2?
    I have seen that you have also conducted a grid frequency monitoring experiment.

    • Well, there are several things you might want to consider.

      1. If you connect an TCXO to an an Uno it will benefit from it a lot of course. Then my library will work.
      2. The sampling hook (e.g. “void sample_input_pin()) will be called once per millisecond. So if you increase a counter in this hook you can use this for monitoring mains frequency. The 1 ms ticks are not phase locked but benefit from the autotuning. Thus you get 1 ppm precision if you let it run for some days in an environment with stable temperature.
  4. Emmanuel A. Neonakis says:

    Regarding the leonardo, I have tried to compile under arduino-1.8.2. None of examples compiled.
    Have you ever successfuly compiled for leonardo? If yes under what version?
    Actually, even for Uno, mb_emulator and super filter failed to compile. What arduino IDE version are you using?
    Thank you.

    • Leonardo is not supported. As you reported even for Uno mb_emulator and super_filter fail to compile. Obviously these examples are broken and need to be fixed. I created a github issue for it. You can track the status here: Thanks for pointing this out.

      • Emmanouil A. Neonakis says:

        Thank you again.
        ‘broken’ is too strong, I am sure they compile properly under a previous compiler version, just telling which one is an appropriate fix for me.
        You state that leonardo is not supported. Do you think it is fundamentally incompatible or just that you do not find it interesting enough to do it?
        I have looked up dcf77_generator.ino. Compilation fails because leonardo lacks timer_2. But after blindly changing all timer_2 references to timer_3, compilation succeeds.
        It also seems that if one is only interested in the baseband signal, the timer_2 is not needed.
        Your advice on porting the generator (and perhaps the library) to leonardo would be welcome.

      • I never had a Leonardo for testing. Hence I can not really support it. I could put your patch into a dedicated branch. If you would be willing to provide test run logs to me which confirm that it actually works, then I would put it into the main line.

      • I added a branch for Leonardo: would you please verify if it works? I do not want to know if it compiles. I checked this. I want to know if it actually decodes. Preferably I would like to have a log of the swiss army debug helper in mode Dm for at least 15 minutes. Even better would be a log in mode Dm for at least 3 days. Reason: this would also verify that the EEPROM functionality works as it should.

        If you or someone else provide the logs I will put it into the main line. Then you would get automatic updates. for Leonardo as well.

  5. Emmanouil A. Neonakis says:

    You are very fast, I was still trying to figure out the timers’ differences between Uno and Leonardo.
    I will order a receiver and will provide the requested information, but I do not expect to receive it earlier than April 17. Crete is a good test place as it is outside the range. (My >17 years old Eurochron clock succeeds to decode but only during the night. At times more than 50 hours elapsed between synchronizations)
    I was actually looking into the dcf generator. Have you figured out if atmega32u4’s timer 3 supports independent determination of pwm’s frequency and duty cycle (via OCR2A and OCR2B) as atmega328 timer 2 does?
    Thank you.

    • You mean Crete Greece? Whoa, I definitely want to have a debug log of 48 hours if possible. This would give me a lot of insights on how the signal behaves under really poor reception conditions. Of course I know that PTB has lots of theory about this. But what I need is real measurements with typical receivers. With regard to the timer –> the autorative source is always the datasheet: Section 14 “16-bit Timers/Counters (Timer/Counter1 and Timer/Counter3)” says it is possible.

      • Emmanouil A. Neonakis says:

        Yes that Crete. You will get the debug log for as many days as you want. But first I must have a receiver. HKW seems not to accept orders from Greece, I may have to settle for the Conrad module.
        Thanks for the pointer. In the dcf77_generator you change the carrier frequency between to values in order to have an average equal to 77.5kHz. That is not recommended for timer 3 channel A, and only output A is brought to a pin. So timers 1 and 3 will have to be interchanged, pwm on timer1 and modulation on timer 3.

  6. Emmanouil A. Neonakis says:

    While waiting for HKW to respond, I borrowed a UNO and programmed DCF77-generator on it (and modified it to output the time signal on pin 7). Then I programmed the Leonardo with the Swiss_Army_Debug_Helper and connected UNO D7 to Leonardo A5. In scope I see the Leonardo receiving the signal pulses with apparently correct durations but they continously shift to the right.
    The clock state is staying in the useless state. Is this a manifestation of the UNO resonator instability? Do you have any suggestion or is the situation hopeless? A log is posted in
    Thank you

    • This is the result of the resonator imprecision. The instability manifests in the fact that the drift varies in speed. With other words: the UNO sucks at timekeeping. The easiest way is to replace the UNO by something with a crystal. If you want to have it the hard way: obviously the emitted frequency is still OK for the receiver. So you only need to get hold of the drift. Since you only want to test you do not need absolutely correct time. So you might want to phase lock the modulator to the receiver. That is: have the receiver generate a square wave of period length 200ms (toggle a pin every 100 ms). This can be done by counting to 100 in the function which samples the receiver input each millisecond. Then after counting to 100 reset your counter and toggle the sync pin.
      Connect some UNO pin to your sync pin on the receiver side.
      Then on the generator side trigger the modulate function by an interrupt on a level change of the sync pin instead of timer1. Alternatively you could use a level interrupt and generate a signal of period length 100ms.

      So yes, this is possible put might not be worth the effort.

      • Emmanouil A. Neonakis says:

        I have implemented your suggestion and it works. Absolutely no drift any more.
        Code and logfile in
        I have noticed that you have deleted dcf77-leonardo_support branch and some files in the master repository are marked ‘Arduino Leonardo support’. Has you already confirmed the code works on leonardo?
        Thank you again!

      • Yes and no. It seems that the code is good. However I am still missing logfiles. I thought that I will get my feedback earlier if it is included in the master branch. That is: I take the word of Aido that the code is OK and see if I get any negative feedback. Your logfile is highly apreciated. However it looks like synthesized data. No noise and absolutely no drift. Will you be able to provide a real world log for me?

  7. Emmanouil A. Neonakis says:

    I thought it was clear that this log was produced by Swiss-Army-Debugger running on Leonardo fed by a UNO running dcf77-generator. In order to overcome the UNO resonator instability, I have modified both programs as you suggested in your April 13, 2017 at 18:54 response above. So yes in a way the log contains synthesized data.
    As HKW does not ship to Greece, I have ordered the Conrad module. They informed me they have shipped it. I hope it arrives next week and works properly. As soon I receive it, I will let you know and post real world logs.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s