With the Crystal Frequency enhancement of my library I can of course fix the biggest issue of my clock library: it does not keep time well if the DCF77 signal is lost. Although this happens rarely due to its extreme noised tolerance, it still will happen (e.g. because DCF77 goes down for maintenance).
First I considered adding a DS3232 RTC but then came to the conclusion that temperature drift is not a big issue as my clock is usually sitting at room temperature. So if the adjustment is roughly good to 1 ppm I would trust this for several hours. The only thing is how to ensure that it is almost always close to 1 ppm.
My solution is to compare the phase of the decoder stage with the elapsed time. The point is that the detector stage (and thus the 1 second tick) is phase locked to the demodulated DCF77 signal. On the other hand the millisecond interrupts and thus the 1/100 second ticks are driven by the crystal clock. Thus it becomes possible to compare the phase of the divided crystal clock with the phase of the demodulated DCF77 signal. This in turn allows me to infer the necessary frequency adjustments.
Lets have a look at the new design and see how it works.
If you compare this with the previous design you notice the new box “frequency control”. This box is implemented in the namespace DCF77_Frequency_Control. As I already explained the basic idea is very simple. The tricky part is the question: how long should I measure? I call the measurement period length “tau”. The unit of measure is 1/100 s. This is because the phase lock in the decoder resolves 1/100 s. It follows that the precision that I can achieve (in ppm) is 1 000 000 * 1/100 s / tau = 10 000 s / tau. Or with to put it another way in order to get down to 1 Hz I would need to measure 160 000 s or almost two days. Unfortunately this implies that under normal conditions this is almost impossible to do. The issue is that crystal frequency depends on temperature and temperature varies some centigrades between day and night. As it turns out this is enough to shift the crystal frequency by some Hertz. On the other hand to short measurements will not allow me to tune to a reasonable precision.
So I experimented quite a lot and came to the conclusion that “bisection” seems a reasonable approach. I start with tau = 2500 s (for a precision of 64 Hz or 4 ppm). After this period I will adjust the crystal in steps of 64 Hz. If the phase deviates by 5 or more ticks (= 1/20 s) then I will immediately adjust and restart the measurement. Eventually it will require only 0 or 1 step adjustment. Once this happens I will double time and adapt the resolution acoordingly.
In case of a 0 step adjustment I will also pretend that half of the (new) measurement period is already elapsed. This way the frequency control will settle faster. This will go on till either tau_max is reached (and thus frequency is controlled to within 1 Hz) or if it does not manage to get better precision anymore (e.g. due to frequency variations caused by the temperature shift between night and day).
In case the phase drifts a lot then the algorithm check if less than tau/2 have elapsed and decreases tau accordingly. If this does not help and the error becomes still bigger it will immediately terminate the measurement and decrease tau.
The constants when to switch are somewhat deliberate but I fiddled and tested quite a lot with them. Testing is pretty time consuming since the time constants are so large though.
namespace DCF77_Frequency_Control {
// tau = length of measurement period in ticks (@ 100 Hz)
volatile uint32_t tau = tau_min;
volatile int8_t precision = precision_at_tau_min;
volatile int8_t confirmed_precision = 0;
// indicator if data may be persisted to EEPROM
volatile boolean data_pending = false;
volatile uint32_t elapsed;
volatile int16_t tick;
volatile int16_t start_tick;
volatile int16_t deviation = 0;
void restart_measurement() {
start_tick = tick;
elapsed = 0;
}
bool calibration_running = false;
void start_calibration() {
calibration_running = true;
restart_measurement();
}
void cancel_calibration() {
calibration_running = false;
}
// get the adjust step that was used for the last adjustment
// if there was no adjustment or if the phase drift was poor it will return 0
// if the adjustment was from eeprom it will return the negative value of the
// persisted adjust step
int8_t get_confirmed_precision() {
return confirmed_precision;
}
// this is always something in between the adjust step contants
int8_t get_target_precision() {
return precision;
}
// 1 tick = 1/100s but this does not matter, the only relevant informaiton is the ratio
// deviation / elapsed_ticks
// this is undefined if elapsed_ticks == 0, it is also pretty meaningless if elapsed_ticks is very small
// elapsed ticks == 0 occurs immediately at the start of a new measurement run,
// it also occurs if the clock is out of sync
void get_phase_deviation(int16_t &deviation, uint32_t &elapsed_ticks) {
if (calibration_running) {
const uint8_t prev_SREG = SREG;
cli();
elapsed_ticks = elapsed;
deviation = deviation;
SREG = prev_SREG;
sei();
} else {
elapsed_ticks = 0;
}
if (elapsed_ticks = 0) {
// necessary even if calibration_running in order
// to avoid glitches right at the start of a run
deviation = 0;
}
}
void debug() {
Serial.println(F("confirmed_precision ? target_precision, total_adjust, deviation(3), elapsed/tau: "));
Serial.print(confirmed_precision);
Serial.print(' ');
Serial.print(calibration_running? '@': '.');
Serial.print(' ');
Serial.print(precision);
Serial.print(F(", "));
Serial.print(DCF77_1_Khz_Generator::read_adjustment());
Serial.print(F(" Hz, "));
Serial.print(deviation);
Serial.print(F(" Ticks, "));
const int16_t deviation_Hz = deviation * precision;
Serial.print(deviation_Hz);
Serial.print(F(" Hz, "));
const int16_t deviation_ppm = deviation_Hz / 16;
Serial.print(deviation_Hz / 16);
const int8_t remainder = deviation_Hz - 16*deviation_ppm;
if (remainder) {
if (remainder>0) {
Serial.print(F("+"));
}
Serial.print(remainder);
Serial.print(F("/16"));
}
Serial.print(F(" ppm "));
Serial.print(F(", "));
Serial.print(elapsed);
Serial.print('/');
Serial.println(tau);
}
bool increase_tau() {
if (precision > precision_at_tau_max) {
tau <<= 1;
precision >>= 1;
return true;
} else {
return false;
}
}
bool decrease_tau() {
if (precision < precision_at_tau_min) {
tau >>= 1;
precision <<= 1;
return true;
} else {
return false;
}
}
void adjust() {
int16_t total_adjust = DCF77_1_Khz_Generator::read_adjustment() - deviation * precision;
if (total_adjust > max_total_adjust) { total_adjust = max_total_adjust; }
if (total_adjust < -max_total_adjust) { total_adjust = -max_total_adjust; }
DCF77_1_Khz_Generator::adjust(total_adjust);
}
void process_1_Hz_tick() {
const int16_t good_deviation = 1; // maximum absolute deviation to increase tau
const int16_t poor_deviation = 3; // deviation to decrease tau
const int16_t bad_deviation = 5; // deviation to decrease tau immediately
if (calibration_running) {
//const int16_t deviation = tick - start_tick;
deviation = tick - start_tick;
const int16_t absolute_deviation = deviation>0 ? deviation : -deviation;
if (elapsed >= tau) {
// measurement finished
adjust();
if (absolute_deviation == 0 &&
precision > precision_at_tau_max) {
// absolute_deviation == 0
// --> no adjustment done
// precision not at maximum resolution
// --> next measurement period will be doubled
// however since no adjustment was done we can
// just continue with the current measurement period
// and thus get faster to the next better resolution
} else {
// adjustment done --> new measurement period required
restart_measurement();
}
confirmed_precision = precision;
data_pending = true;
if (absolute_deviation <= good_deviation) {
increase_tau();
} else
if (absolute_deviation <= poor_deviation) {
// stay with current precision
} else {
if (!decrease_tau()) {
confirmed_precision = -precision;
data_pending = false;
}
}
} else { // elasped < tau
if (absolute_deviation >= poor_deviation) {
while (elapsed + elapsed < tau && decrease_tau()) {
// decrease_tau as much as reasonable, notice that decrease_tau()
// will return true if it decreased tau
// also notice that it allows for coarser adjustments
}
}
if (absolute_deviation >= bad_deviation) {
// tau was already decreased but we drifted > 4/100 s this measurement period
// --> drift is way to high, terminate the measurement immediately and
// adjust as if the measurement was completed
// --> adjustment is probably not sufficient but we get a new measurement
// period and thus the change to react earlier, also since tau will
// be decreased larger adjustment steps become possible
adjust();
restart_measurement();
decrease_tau();
confirmed_precision = -precision;
data_pending = false;
}
}
tick -= 100;
}
}
void process_1_kHz_tick() {
static uint8_t divider = 0;
if (divider < 9) {
++divider;
} else {
divider = 0;
// ticks will increase every 10 ms = @ 100 Hz
++tick;
++elapsed;
}
}
// ID constants to see if EEPROM has already something stored
const char ID_u = 'U';
const char ID_k = 'P';
void persist_to_eeprom(const int8_t precision, const int16_t adjust) {
// this is slow, do not call during interrupt handling
uint16_t eeprom = eeprom_base;
eeprom_write_byte((uint8_t *)(eeprom++), ID_u);
eeprom_write_byte((uint8_t *)(eeprom++), ID_k);
eeprom_write_byte((uint8_t *)(eeprom++), (uint8_t) precision);
eeprom_write_byte((uint8_t *)(eeprom++), (uint8_t) precision);
eeprom_write_word((uint16_t *)eeprom, (uint16_t) adjust);
eeprom += 2;
eeprom_write_word((uint16_t *)eeprom, (uint16_t) adjust);
Serial.println(F("persisted to eeprom"));
}
void read_from_eeprom(int8_t &precision, int16_t &adjust) {
uint16_t eeprom = eeprom_base;
if (eeprom_read_byte((const uint8_t *)(eeprom++)) == ID_u &&
eeprom_read_byte((const uint8_t *)(eeprom++)) == ID_k) {
uint8_t ee_precision = eeprom_read_byte((const uint8_t *)(eeprom++));
if (ee_precision == eeprom_read_byte((const uint8_t *)(eeprom++))) {
const uint16_t ee_adjust = eeprom_read_word((const uint16_t *)eeprom);
eeprom += 2;
if (ee_adjust == eeprom_read_word((const uint16_t *)eeprom)) {
precision = (int8_t) ee_precision;
adjust = (int16_t) ee_adjust;
return;
}
}
}
precision = 0;
adjust = 0;
}
void read_from_eeprom(int8_t &precision, int16_t &adjust, uint32_t &tau) {
int8_t ee_precision;
read_from_eeprom(ee_precision, adjust);
tau = tau_max / precision;
}
// do not call during ISRs
void auto_persist() {
// ensure that reading of data can not be interrupted!!
// do not write EEPROM while interrupts are blocked
int16_t adjust;
int8_t precision;
cli();
if (data_pending && confirmed_precision > 0) {
precision = confirmed_precision;
adjust = DCF77_1_Khz_Generator::read_adjustment();
} else {
data_pending = false;
}
sei();
if (data_pending) {
int16_t ee_adjust;
int8_t ee_precision;
read_from_eeprom(ee_precision, ee_adjust);
if (confirmed_precision < abs(ee_precision) || // precision better than it used to be
( abs(ee_precision) < 8 && // precision better than 8 Hz or 0.5 ppm @ 16 MHz
abs(ee_adjust-adjust) > 8 ) || // deviation worse than 8 Hz (thus 16 Hz or 1 ppm)
( confirmed_precision == precision_at_tau_max && // tau_max > 1 day thus it is
abs(ee_adjust-adjust) > 2 ) ) // acceptable to write more often
{
cli();
const int16_t new_ee_adjust = adjust;
const int8_t new_ee_precision = precision;
sei();
persist_to_eeprom(new_ee_precision, new_ee_adjust);
}
data_pending = false;
}
}
void setup() {
int16_t adjust;
uint32_t ee_tau;
int8_t ee_precision;
read_from_eeprom(ee_precision, adjust);
if (ee_precision < precision_1_ppm && ee_precision > 0) {
// we assume that at startup we will never be better than 1 ppm,
// no matter how good the previous calibration was
ee_precision = precision_1_ppm;
} else {
ee_precision = precision_at_tau_min;
}
ee_tau = tau_max / ee_precision;
cli();
tau = ee_tau;
precision = ee_precision;
sei();
// start with negative values for precision
// so we know the absolute value of the precision but
// still have an indicator that it was read from EEPROM
confirmed_precision = precision > 0? -precision: precision;
DCF77_1_Khz_Generator::adjust(adjust);
}
}
If you look into the code you will also notice that it contains some access to EEPROM. Since the frequency control takes so long to converge I found it reasonable to memorize the last good values in EEPROM. THis is trickier as it looks because EEPROM write takes way to much time for being processed during interrupts.
My solution was to store the to be persisted values in RAM and have the auto_persist() function write them to EEPROM later outside the interrupt functions. In order to make this transparent to the current users of the library I put this feature into the get_current_time() function. There is one catch though. If you handle the complete time output with the output handler and the read_current_time() function then you need to tick the auto_persist function every once in a while. I suggest to call it at the start of loop() but your mileage may vary depending on what you intend. If you are after something really tricky just drop me a mail 🙂
One final comment on EEPROM wear. Of course I know how to implement wear leveling. However I decided to not implement it here. My analysis shows that the persistency will be called about every 12 hours or so. In addition I will only persist if the maximum precisions is reached (and thus tau > 2 days) or if the deviation from the former value is >1 ppm. So I figure it would take >100 years to wear out the EEPROM. Thus it would be pretty pointless to add wear leveling here.
Blinkenlight 