DCF77 Library

This library collects the hard part of The Clock into a reusable library. With this library you can have the same extraordinary noise tolerance for your own clock projects. In case you wonder what exactly “extraordinary noise tolerance” means read on here.

Before we go into the details have a look at the architecture overview picture. It is somewhat similar to the architecture of “The Clock”.

The_Clock_Library_Architecture

The differences are mainly due to the fact that I only wanted the decoder stuff in the library. Unlike other DCF77 libraries this library is not bound to any specific means of signal acquisition. Especially it is not bound to any pin at all. Instead it executes the inputprovider callback each millisecond. The input provider must be of type typedef uint8_t (*input_provider_t)(void);. That is it must have no arguments and return either 0 or 1 depending on the current state of the DCF77 signal.

Once per second it will execute the output handler callback and pass the current time to it. The output handler if of type typedef void (*output_handler_t)(const time_t &decoded_time);

These callbacks can be set with the functions

DCF77_Clock::set_input_provider

and

DCF77_Clock::set_output_handler.

The input provider is mandatory the output handler is optional. If you do not set the output handler the clock will work anyway. However the output handler is very convenient in order to set a display in sync no matter what the main loop of your sketch is currently processing.

Alternatively you can retrieve the current time by calling the functions

DCF77_Clock::void get_current_time(time_t &now); // blocking till start of next second

and

DCF77_Clock::void read_current_time(time_t &now); // non-blocking, reads current second.

This sounds slightly complicated but it is very simple in practice. In order to illustrate the concept I included some examples with the library. You can download everything from github.

Examples

The library includes some examples to get you started.

DCF77 Scope

The DCF77 Scope does not rely on the library at all. It is an analysis and troubleshooting tool. If this tool does not give reasonable results the library will not work.

Simple Clock

The Simple Clock implements a rudimentary DCF77 clock to exhibit the basic functionality of the library.

The Clock – Library Version

The Clock – Library Version is the library based version of The Clock. This is the code that started it all.

Meinberg Emulator

The Meinberg Emulator emulates a Meinberg DCF77 Clock. I implemented it because it was easy and because there are some NTPD Meinberg Drivers out there.

Super Filter

The Super Filter is intended as a drop in enhancement for existing DCF77 projects. Just put the Super Filter between the DCF77 module and your existing gadget and you will have all the benefits of my library without touching your code. It basically decodes the signal and re-synthesize it. Thus you will get a 100% clean signal.

Download

You can download the library and the examples from github.

Gallery

Brett Oliver uses my Library for his incredible DCF77 master clock.

Hardware Incompatibilities

My library will not work with a “modern” Arduino Uno. The Uno utilizes a resonator instead of a crystal. Since a resonator has a less stable clock this in turn would decrease the noise tolerance significantly. It also would require different constants for the phase lock. Since my goal is to push the noise tolerance to the limits I will offer no support for this platform. You have several options:

  1. get hold of an “old style” Arduino like e.g. the Blinkenlighty (which would also help to fund my blog).
  2. replace the Uno’s resonator with a crystal or a crystal oscialltor (requires some soldering skills)
  3. dig into my library and modify the time constants (possible, but gives poorer results, I neither recommend nor support this approach)

Software Incompatibilities

The initilization code for this library will stop timer 0 interrupts. Thus the Arduino’s millis() function will not work properly anymore. This is usually not an issue since my library is already keeping time, thus there should be no need for a redundant time keeping mechanism. If you absolutely require the Arduino’s standard time keeping you may activate it by setting TIMSK0 accordingly. However you need to understand that I never tested this setup, thus it might decrease the library decoder performance. Only reactivate this if you understand the implications of race conditions.

The library also reprograms timer2. Thus anything that relies on control of timer2 (e.g. msTimer2) will interfere with my library.

The library works fine with my Arduino 1.0 installations on Ubuntu. However for higher versions of Arduino it compiles but fails to acquire a signal lock. The nasty reason for this is that the Arduino guys bundle an outdated version of avr-gcc with the newer versions of Arduino. I have no clue why this is the case. The newer versions of gcc create significantly faster code. Since my library requires lots of computations and has only a tight time budget this matters a log. So if you consistently fail to get a lock with this library please check your compiler version. If it is before 2010 then your compiler is definitely to old.

If you are using windows then you are lucky. As Werner Gaulke pointed out there exist detailed instructions by Andy Brown on how to install a newer avr-gcc.

Pin Mappings

Some readers wondered to which pins my library maps. The answer is: none. This library was designed to be completely pin agnostic. This of course rises the question how this library learns about the signal. The answer is that YOU have to provide the code for sampling the desired input pin. An example of how to do this is in the simple clock implementation. Watch out for the function “sample_input_pin()”.

Some readers also wondered what the meanings of the constants in my examples are. Of course I described them somehwere in my blog. But if you do not want to go through all of my articles here is the list of some standard constants that I use.

If these constants do not suit your needs, just change them. The library has no implicit dependency to any pin at all. So you can use any pin as long as it is not in use for something different.

name typical value comment
dcf77_analog_sample_pin 5 input pin for analog sampling
dcf77_sample_pin 19 input pin for digital sampling, notice D19 == A5
dcf77_inverted_samples 1 0 or 1 depending on your DCF77 module
dcf77_analog_samples 1 1 => analog sampling, 0 => digital sampling, Blinkenlighty needs analog samples see comment below
dcf77_monitor_pin 18 displays the signal the software received, common Arduinos might want to use 13 because this is connected to the only LED

With regard to the analog vs. digital sampling. Analog mode will work with any Arduino I know. However it consumes slightly more resources. For the Blinkenlighty it is the only admissible mode if you do not want to disconnect the LED bar. So set this to 1 for the Blinkenlighty. For all others you may want to set this to 0.

Depending on this mode either the sample pin or the analog sample pin will be evaluated. Of course these constants may point to the SAME pin. In digital mode the digial sampling pin is used in analog mode the analog sampling pin is used. The choice of the mode has absolutely no effect on the noise tolerance. In the end it only controls the threshhold between a “low” and a “high” signal.

If you are unsure about the pin mappings use the DCF77 scope with your pin mappings to verify if you got them right.

24 Responses to DCF77 Library

  1. I’m currently looking into adopting the library in one way or another. More specifically I want to improve the decoding robustness for a Wordclock project [1]. However, I’m not sure as to whether your library is suitable for our situation, because we use the internal RC oscillator of an Atmel ATmega168/328.

    If it shouldn’t be possible to adopt your library directly, it would be great to get some advice for implementing a solid and robust filter on our own. Knowing what you know now (after having spent a couple of months dealing with all of this), what would you personally suggest is a good compromise between spent effort (time, code size) and actual improvements. Personally I’m of the opinion that the exponential filter approach looks the most promising, because it is relatively straight forward to implement and according to your blog entry improves the decoding capability by a lot.

    I have another couple of questions, which are more subjective and I’m asking them just out of interest:

    – Have you ever dealt with DCF77 modules that would only return a low level in cases they are interfering with some PWM signal? I’ve two modules that are permanently low in some instances, so filtering in software won’t even work. What can be done about this?

    – Do you know about the capacitor mod for Conrad DCF77 modules proposed in the current “c’t Hardware Hacks”? They suggest to add a decoupling capacitor parallel to a diode on the module, which seems to improve the quality of the signal. Personally I haven’t observed any improvement, so I’m not sure as to whether this makes any difference at all.

    – Have you noticed any difference in the modules from Pollin, Conrad and Reichelt? Only recently I’ve made the experience that the ones from Pollin seem to be more robust. Given that they only cast half as much as both of the other options, I’m really confused. Unfortunately they don’t work with TTL (5 V) logic, so you need some external components for the level shifting. What are your thoughts on this? I’m not talking about the industrial receiver(s) you have worked with here ;).

    [1]: https://www.mikrocontroller.net/topic/156661

    • The *best* solution regarding time and spent effort is to use an out of the box solution. For example HKW sells modules with internal clock. Taking opportunity cost into account this is the most efficient solution.

      Now if you want to follow the diy path the next best thing is to get a reasonable receiver module and a big antenna. I compared several DCF77 receiver modules. Again the professional solution wins. Especially because it will work with a bigger antenna (bigger is better here).

      The HKW module (like most of the professional modules) has a wider bandwidth. The cheap modules have a narrower bandwidth. The reasons why are intricate but this has the implication that the professional modul will pick up more (sic!) interference. I suspect that this is the reason why the professional modules seem to have better automatic gain control. The AGC seems to be the reason why the cheaper modules will flatten out if there is to much noise.

      Now with regard to the optimal filter approach. I would suggest to add a crystal oscillator. They come out of the box as complete modules and will cost you only one pin. Then you can use my super noise resistant library. In the last test I laid an active Smartphone on the Antenna. The HKW module’s signal went mad but the module continued to demodulate the crap. The library decoded the signal within ~30 minutes although there was no minute with at least 10 bit errors.

      If you do not want to go for a crystal and not for a smarter DCF module then I see only two more options:

      1) Use the exponential filter approach . But remember that the libary can deal with 15 dB worse SNR.

      2) Use a diy smart module. That is use another 328 with my library in it and use it to locally synthesize DCF77 again. I am very sure that this would even beat the HKW smart decoder module. If you are interested in this approach I would publish a sketch for it.

      With regard to filtering of powerline noise forget about the magic capacitor approach. The only reliable solution is to nail down the root cause. Get an oscilloscope or find one who owns an oscilloscope and determine what is going on. Once this is understood fix the root cause (e.g. add suitable caps or inductors or replace the power supply or whatever else might fix it). Then measure again and verify that you solved it. Anything else is just searching in the dark == a big waste of time.

  2. Unfortunately, adding an oscillator to the existing circuit is not really an option. Building a secondary board and connecting it via SPI / I2C on the other hand might be worthwhile. On the other hand the most annoying problem for me is that the modules won’t return anything when they are in a high noise environment, so a secondary circuit will not necessarily help with that. However, I’m not sure what exactly is going on, because looking at the powerline with an oscilloscope does not look suspicious. Maybe some EMF issues, but I neither have the capability to examine this, nor do I know how to fix it.

    Have you planned (or are you interested in) building a module (based on AVRs) hosting your code? I think this might be a good idea after all, so other projects could simply ask for the decoded time (and some status information) via SPI / I2C and/or any other arbitrary protocol (1-Wire).

    • Yes I put some thought into it but due to EEW regulations in Europe this would be prohibitive expensive, so I ditched this idea.

      • Do these regulations really apply if you sell a PCB for other hobbyists? Maybe you could sell it as a kit? It seems to work fine for other projects …

      • Unfortunately they apply. I have registered for business (because that way I can order stuff that I would not get otherwise). However this also implies that authorities can assume that I know what I am doing. I already analyzed the issue and EEW definitely applies to me. This was the reason why the Blinkenlight Kit was first sold in the US. It was also the reason why I teamed with Franzis to sell the shield in Germany. I would have to check if Franzis or some other distributor would be willing to pick up my DCF stuff. IMHO the easiest way would be to take some prefabricated Atmel board that features a crystal. These are available for 15 Euro or less. Then just flash my code to it and connect whatever DCF77 module you want. I suggest HKW modules which are around 20 Euro. Pollin modules are also OK for their price but I will always go for quality. Now suppose I would sell this as a kit. Then we would end up around 50-60 Euro because I also have to account for the associated risk. Would there be any demand at this price point? I doubt this. Reason: IMHO the only demand is from people at some expert level. These people could easily source the stuff wherever they want and just flash my library to it.

  3. Karl says:

    I’m trying to get your DCF-77 Library working with a “Freaduino MEGA2560 White Color” Board, which features a crystal and has a reasonable clock deveation of about 200ms every 3600 seconds resulting in a shift of 888 Hz. I use an EM2S-DCF Module and the AFET100-77,5-DD antenna from HKW-Elektronik.

    I’m on a Mac and tried with the available Arduino IDEs. With Arduino 1.0.0 and Arduino 1.0.5 (with avr-gcc version 4.3.2) i can compile and run the DCF77_Scope example which shows the beforementioned clock deveation.

    The “Simple_Clock” and “The_Clock” sketches also compile and run on the board, but i never get the time output.

    When i use Arduino 1.5.7 beta with “Simple_Clock” i get the following error when compiling:

    /Users/karl/Documents/Arduino/libraries/dcf77_library/dcf77.cpp: In function ‘void DCF77_Encoder::_ZN13DCF77_Encoder14advance_minuteERN5DCF7711time_data_tE.part.4(DCF77::time_data_t&)':
    /Users/karl/Documents/Arduino/libraries/dcf77_library/dcf77.cpp:3142:1: internal compiler error: Bus error: 10
    }
    ^
    libbacktrace could not find executable to open
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See for instructions.

    The revisions.txt of Arduino 1.5.7 beta says “Upgraded AVR toolchain: gcc 4.8.1, avr-libc 1.8.0”.

    I read your post on the arduino-forum regarding the avr-gcc issues on linux, but couln’t figure out if i have the same problem:

    http://forum.arduino.cc/index.php?topic=218014.0

    Any ideas how i can get this working?

    (The Hardware works flawlessly with the DFC-77 library of Thijs Elenbaas, but not in the specific place where i need it to be working, because of bad signal quality)

    • 4.3.2 is definitely outdated. I never tried 4.8.1. I use 4.5.3. I suggest you give 4.5.3 a try and see how if it compiles.

      • ZT says:

        I’m doing some tests using Arduino 1.5.7 Beta, beacuse avr-gcc 4.7.0 crash compiling everything for mega2560 and I can’t find 4.5.3 for windows. I get this error:

        Arduino:1.5.7 (Windows XP), Scheda:”Arduino Mega or Mega 2560, ATmega2560 (Mega 2560)”

        \Arduino\libraries\dcf77_library\dcf77.cpp: In function ‘void DCF77_Frequency_Control::process_1_Hz_tick(const DCF77::time_data_t&)':
        \Arduino\libraries\dcf77_library\dcf77.cpp:2971:70: error: using temporary as lvalue [-fpermissive]
        ((DCF77::time_data_t)decoded_time).leap_second_scheduled = true;
        ^
        \Arduino\libraries\dcf77_library\dcf77.cpp:2979:70: error: using temporary as lvalue [-fpermissive]
        ((DCF77::time_data_t)decoded_time).leap_second_scheduled = leap_second_scheduled;
        ^

        Searching arount internet I’ve found some indication to add -fpermissive to compiler flags. Adding this flag the compiler crash and tell to report the crash as gcc bug.

      • I have no idea why avr-gcc 4.7.0 crashes. But it is definitely a compiler issue. Arduino ships by default with an outdated compiler. I am currently using avr-gcc 4.5.3 Would you mind to try 4.5.3? If you tell me the settings for the “inverted mode” I could compile a “debug helper” for you board. Did you need to modify anything for the timer setup or did it work? I am asking because I have no Atmega 2560 around. So if some modifications are needed for the timing I would need this piece of code.

        If there is no 4.5.3 for windows I can only recommend to boot Linux from an USB stick. Although Linux might be unfamiliar to you the Arudino IDE looks exactly the same.

      • ZT says:

        I can not try on the real card at the moment (failure of power supply and failure of all attached devices ….. But replacements are coming.), So I’m just compiling the code. I’ve never tested the library on Mega2560 Because I was never able to complete the scketch.I think that could be nice modify the library to use automatically timer4 or 5 or 6 if the target is Mega2560 (so standard arduino funtions aren’t compromised). I’m also looking for a way to compile using beta arduino ide. The error “using temporary as lvalue” is not really a compiler issue but a new working mode of gcc that need some code changes.

      • I agree that it would be nice. However I do not have a 2560 board, so I can not test this. I suggest that you check out my code from github here. Then add the necessary stuff. Once you tested it issue a pull request and I will integrate it into the code base. The only stuff that you need to change is the implementation of void init_timer_2(). In addition the ISR(TIMER2_COMPA_vect) has to be replaced by a vector that is suitable for your timer. Both reside close to the end of dcf77.cpp. You do not need to deal with macros to find out which board you are on, I would add this part of the code.

  4. karlkliem says:

    I followed Andy Browns instructions and installed avr-gcc 4.7.0 with Arduino 1.0.0 on Windows, which you were refering to as well. With the Scope sketch the serial monitor shows the following:

    1, +———+———+———+———+———+———+———+———+———+———
    2, 52——–+———42——–+———42——–+———42——–+———42——–+———
    3, 42——–+———42——–+———42——–+———42——–+———42——–+———
    4, 42——–+———42——–+———42——–+———42——–+———42——–+———
    5, 42——–+———42——–+———42——–+———42——–+———42——–+———
    6, 42——–+———42——–+———42——–+———42——–+———42——–+———

    […]

    And the Simple_Clock sketch still does not sync.

    As i also didn’t find any information about how to install avr-gcc 4.5.3 (or a similar version) with the Mac Arduino IDE (and my attempt with Ubuntu 14.04. also installed the latest version of avr-gcc) i’m about to give up and order a HKW-Elektronik FUM1 FSK module, which would do the decoding for me.

    Though it would be great if you could provide information about how to get a setup for compiling your DCF77 library on Mac, Windows and Linux. Greetings and thanks!

    • Of The scope shows that you are NOT picking up any DCF77 signal. There are several possible reasons.

      • There is something close to the antenna that interferes with DCF77
      • The module has an open collector output and you are missing a pullup
      • The wiring of the module is wrong, e.g. missing ground connection

      Hence I would be more than surprised if it would pick up anything at all.

  5. karlkliem says:

    Finally i get the sync! Thanks!

    Although it’s strange. I had the same circuit connected to the same computer and uploaded the same sketch. At first with the Mac Arduino IDE and then with the Windows Arduino IDE (inside a Parallels Desktop VM), and it didn’t work. Now i tried again copy/pasting the sketch from one IDE to the other and it worked. Finally!

    If i find the time i will try to get avr-gcc 4.5.3 running with the Mac Arduino IDE. In case i succeed i’ll make a post.

  6. karlkliem says:

    It’s going to be in a building in Vienna, which i haven’t seen myself yet. Until now i just sent a test setup with the Conrad Elektronik DCF module and the 100mm HKW antenna and an Arduino sketch building upon the library of Thijs Elenbaas. They could get a signal in one floor of the building which has windows, but not in the floor where the clock will be installed. The walls in that floor seem to be made out of reinforced concrete. My plan is to deliver another test setup with your library and the HKW module now.

  7. Andreas says:

    Hello,

    I started to test your DCF77 lib with the noise reduction algorithm.
    Works fine!. But it uses the Pin A5 and I want to use this pin for I2C.
    All tries to connect to another pin failed…. Does it work with a different pin and how?

    Thanks, Danke. Andreas

    • The library has no intrinsic pin binding. All binding happens by the calling program. It can be bound to any pin as long as this pin is not in use otherwise. Just change the constants accordingly. I will make the section “pin mappings” a little bit more explicit on this topic.

  8. ZT says:

    Hi, I’m playing round your library to develope a clock. I need to play tones on a piezio speaker, but all library use millis(), and this funtion was compromised by your library. I’m not a good programmer bue I see that the library has an internal function called 1000 times as second, so is simple to add a method that do the same thing as original millis(). Are there a way to override millis() in a transparent way so all libraries that use this function still work?

    • Hi, as far as I know there is no such way. To achieve this it would be necessary to somehow write to the variable volatile unsigned long timer0_millis = 0 of wiring.c Alternatively it would be necessary to replace unsigned long millis() from wiring.c.
      I am not aware of any way to achieve this “transparently” without actually patching wiring.c. If you would give more details on what you are actually trying to achieve (more details means CODE) then maybe I can find a better solution.

      • ZT says:

        I haven’t the code here but the example is simple to explain.
        I have to use this library: http://playground.arduino.cc/Code/ToneAC
        This library generate a tone (note), forever or for a specific time. To track the note duration it use millis(). I could use the funcitons toneAC()/noToneAC() to start/stop the notes and generate the melody, but I need some calls that give me a time with at least 5ms of resolution, better if 1ms. So a function, on your library, that return a values as millis() do or that return current time with 1/1000s resoultion can be a good workaround.

  9. Have a look at the http://blog.blinkenlight.net/experiments/dcf77/simple-clock/ example. The sample_input_pin method is called once per millisecond. Thus if you could define a global counter
    volatile unsigned long my_millis;
    Then increment this counter in sample_input_pin, e.g. put
    ++my_millis; as the first line into sample_input_pin
    Then use something like

    unsigned long read_my_millis() {
    const uint8_t oldSREG = SREG;
    cli();
    const unsigned long m = my_millis;
    SREG = oldSREG;
    return m;
    }

    To read the desired milliseconds. Do not use my_millis, otherwise you will get some nasty race condition.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s