ohlcdb

Gets price candles from the cTrader Open API into PostgreSQL, and keeps them up to date.

Each run downloads, for every configured pair and period, the candles that arrived since the last one stored, then exits. Run from cron it keeps the database a few minutes behind the market, and a run missed while the host was down is made up by the next one.

Requirements

• A cTrader account, and an application registered on openapi.ctrader.com with its client id and secret. Its Playground gives the first pair of tokens: the accounts scope, view only, is all ohlcdb needs.
• PostgreSQL, reachable from the host that runs ohlcdb.
• Python 3.11 or later with psycopg and websockets, and mikefarah's yq (yq-go on Alpine).

Install

On Alpine, apk add ohlcdb. The package creates the ohlcdb user, installs /etc/ohlcdb/config.yaml, and leaves schema.sql in /usr/share/ohlcdb. The package is built from build/alpine/APKBUILD.

Setup

1. Configure, as root:

   ohlcdb config
   

   It asks for each setting and checks it against cTrader and PostgreSQL before saving, then offers to collect every 15 minutes. When the database is not there yet, it ends by printing the commands that create it, filled in with the host and names just given, for a PostgreSQL superuser to run.

   The questions follow the file: the database, then each source in turn - cTrader, Dukascopy, Databento - then rollup. Each source is behind a yes or no, and saying no to one asks nothing about it and writes none of its settings, leaving whatever the file already says about it alone. So a setup with one source is a handful of questions, and changing a date in another later does not mean answering for all three.

   schema.sql takes the names from three psql variables, db, owner and reader, which default to ohlcdb, ohlcdb and ohlcdb_ro. The owner holds the data; the reader can only read it, and is the one to query with.

2. Download the history once, as far back as wanted:

   ohlcdb -s 2015-01-01
   

   Months before the broker's first candle go by with nothing received.

   Alternatively, set start in the configuration: a pair or period with nothing stored then begins from that date on the next run, periodic ones included, and so does any pair or period added later.

Commands

ohlcdb config [show]                                              set the configuration, or print it by section
ohlcdb pair [list] [-b BROKER] [-e] [--contracts]                 the pairs in the database, or only the empty ones
ohlcdb pair search TEXT                                           look up the account's pairs
ohlcdb pair add PAIR...                                           add pairs to the configuration
ohlcdb pair remove [-b BROKER] PAIR... [--candles]                remove them, keeping what was collected
ohlcdb [collect] [-s DATE] [-u DATE] [-p PAIR]... [-t PERIOD]...  download from every source configured
ohlcdb ctrader [-s DATE] [-u DATE] [-p PAIR]... [-t PERIOD]...    the broker's candles, then a share of the ticks
ohlcdb ctrader ticks [-b REQUESTS] [-s DATE]                      only the ticks, with no budget unless given
ohlcdb ctrader accounts                                           list the accounts the token reaches
ohlcdb ctrader token [-n]                                         refresh the token, asking for a pair when needed
ohlcdb dukascopy [-b DAYS] [-s DATE] [-u DATE]                    only the history Dukascopy publishes
ohlcdb databento [-b SPEND] [-s DATE] [-u DATE]                   only what Databento sell, cost checked first
ohlcdb roll [-s DATE] [-p PAIR]... [-t PERIOD]...                 build the longer candles from the minutes
ohlcdb analyse [-b BROKER] [-p PAIR] [-s DATE]                    measure the sessions, writing nothing
ohlcdb enable [-e MINUTES]                                        collect periodically
ohlcdb disable                                                    stop collecting periodically

config show prints the file the way the file is written: what is not in it first - where it is, when it runs - then the database, then a block per source with a key to a line, then rollup, which belongs to all three. A source naming no instrument is (off) and nothing else, since none of its other settings does anything. Secrets are said to be set, never shown.

A source that fails is that source's failure and not the run's: the others carry on, the longer candles are built from whatever did arrive, and the command ends by saying which one did not finish - so the log shows it and the exit status is not a success. The free history is not held back by a bill, and a run that stored minutes does not leave them without their hours until the next one.

A source is a noun: on its own it collects what it has to add, and what belongs to it alone hangs underneath. The ticks, the accounts and the token are cTrader's and nobody else's, which is why they are written where they are - the free history and what Databento sell have neither.

-c FILE picks another configuration file; every command has its own -h. A pair added later starts empty: ohlcdb -s DATE -p PAIR downloads its history without fetching the other pairs again. -u DATE stops at a date, which fills a gap without fetching again what is already stored.

-t PERIOD does for candle lengths what -p does for pairs, and each source again takes its share: the broker is asked for the lengths in ctrader.periods, the ones in rollup are built, and Dukascopy and Databento fetch only when M1 is among them, minutes being all they have. ohlcdb -t H1 -s DATE fetches the broker's hours and builds everyone else's without downloading a minute. A length in neither list is refused rather than fetched, so -t narrows the configuration and never adds to it. The ticks are left for a run that did not choose: a tick is not a length of candle.

pair remove takes the name out of whichever source names it - a broker's pair, a Dukascopy instrument, a Databento symbol - and out of the database too, where nothing was ever stored for it. A name typed by mistake would otherwise leave a pair behind that the page goes on offering with nothing to draw; one that has candles keeps them, and keeps the row they hang from. Nothing here can delete a stored candle: the foreign keys from bars and ticks refuse it, which is why this needs no flag to guard it.

pair list, which is also what a bare ohlcdb pair does, is the other side of that: one line per pair in the database, what it holds, and whether a source still names it. The page leaves out the pairs holding nothing - there is no kind of bar to draw for them - so this is where they are visible, and -e shows those alone.

The contracts of a futures product are one line, the product's, saying how many they are and what they cover between them; --contracts lists them one by one, with their product where CONFIGURED would be. That column would otherwise read no for every one of them - rightly, since GC.FUT is what the configuration names and GCZ24 is not - and a hundred and twenty false alarms would bury the one pair that really is named by nobody.

The two halves are independent, and either alone is enough: a pair dropped from the file long ago still has its row, and that orphan is usually what somebody removing a pair can see. Only a name that is in neither is an error.

One instrument can be two pairs: XAUUSD is a broker's and a Dukascopy one, and two sources can name the same string. A name that does not pick out one pair is refused rather than guessed at, with the feeds that answer to it printed, and -b says which - the broker's name, as pair list shows it and as analyse already takes it. Everything is asked before anything is written, so a refusal leaves the file and the database as they were.

--candles is that refusal answered on purpose, for the pair collected by mistake that has been collecting for a week. It says what the pair holds and asks, once for each, and the three deletes - ticks, candles, then the pair - go together in one transaction. It needs a terminal, and checks for one before touching the configuration, so a run from cron leaves everything as it was.

Configuration

/etc/ohlcdb/config.yaml, commented. ohlcdb config and ohlcdb pair rewrite it through yq, which keeps the comments but not blank lines.

Where the candles go first, then a section for each source they come from - ctrader, dukascopy, databento - then rollup, which is what is built out of them whatever brought them. A source with no instruments named in it is off, and nothing of it is ever contacted.

Each source keeps its own pairs and its own start date inside its own section. They were at the top of the file when cTrader was the only one, where pairs and start needed no owner; a configuration written that way is not read as it stands - the run stops and says what to move where, since the alternative is finding no pairs and quietly concluding there is nothing to collect.

A pair carries what the broker says about it: its digits, where it puts the pip, and how many units a lot holds - a hundred ounces of gold. They are asked for once, when the pair is first stored; a pair that predates them is filled in by the next collection, at the cost of one request. The lot size arrives in hundredths, as volumes do there, and is stored in units.

Where there is no broker to ask, the prices answer instead. Databento's digits are counted off them - the fewest decimal places that cover ninety-nine hundredths of what arrives - and the pip is put at the last of those, 0.10 on the gold future, which is that contract's tick as well. It is not the tick on every contract, and it is written only where nothing is known yet: a figure corrected by hand stays. Without it a size in pips means a hundredth on one feed and a whole dollar on another, and the same entry in the page's row of sizes draws two incomparable charts.

The tokens are not in it. A refresh invalidates the pair it is given and hands back a new one, so a pair from the Playground is good for exactly one refresh, and the only live pair is the one in /var/lib/ohlcdb/tokens.json. A copy in the configuration file would be a spent pair pretending to be usable, so there is none.

ohlcdb ctrader token is what makes that file: it uses the stored pair if it still works, and asks for one from the Playground when there is nothing stored or the server turns the stored one down. What is typed is never written as it is — refreshing it is what proves it — and only what comes back is stored, with the pair it replaces kept beside it as tokens.json.old. So a pair cTrader refuses changes nothing. ohlcdb config does the same step when its cTrader section is answered, and it is the only thing there that touches the token file - saying no to that section leaves it as it was. And ohlcdb ctrader token -n asks for a pair even when the stored one works, which is how to move to another application or another Playground pair.

From then on it looks after itself: ohlcdb refreshes the pair a week before its month runs out, so a collector that runs at all keeps the chain alive. The other commands never ask — cron has no one to answer — and stop with a message naming ohlcdb ctrader token instead.

Periodic collection

ohlcdb enable adds a line to the crontab of the user who owns /var/lib/ohlcdb, which the package makes ohlcdb, so the collector never runs as root. Its messages go to /var/log/ohlcdb/ohlcdb.log, one timestamped line per pair and period. The package rotates the log monthly when logrotate is installed.

Data

Table	Holds
brokers	a broker, as cTrader names it
feeds	a price stream: broker, live or demo, account type, what kind of market it is and who sold it
pairs	a pair of a feed: description, digits, pip position, lot size
bars	the candles: bid prices, a volume, UTC opening time
ticks	the quote updates themselves, bid and ask, by the millisecond

The view candles joins them, for queries written by hand:

SELECT ts, open, high, low, close
FROM candles
WHERE broker = 'FPMTrading' AND environment = 'live' AND account_type = 'Raw'
  AND pair = 'XAUUSD.raw' AND period = 'M1'
ORDER BY ts DESC
LIMIT 10;

The account type is what the configuration says, compared letter for letter: Raw and RAW would be two feeds.

What volume counts is the feed's business. From cTrader it is the number of quote updates of both sides, a whole number; another feed may report what was traded on it, which has a fraction. The column takes either, and a query that adds the volumes of two feeds together is adding two different things.

Ticks

A tick is a quote update, not a trade: there is no exchange behind a CFD, only the broker's bid and ask. ticks keeps them by the millisecond, a side at a time, in a month's partition each.

The candles hold the same story summarised, and cTrader builds them from both sides at once: a candle samples the bid at every update of either side, so a minute whose first update is on the ask opens where the minute before closed, and volume counts the updates of both sides rather than the ticks of one. Rebuilding a day of candles from the ticks that way matched the server's own, minute for minute, prices and volumes alike.

What the ticks add is the order of events inside a minute: with candles alone, a minute that touched both a stop and a target cannot say which came first.

Gold runs about 250 thousand ticks a side a day, so catching up on months of them takes thousands of requests. ticks.budget is how many a run spends; the rest waits for the next run, and the candles are never held up.

Dukascopy

Dukascopy publish their history for nothing: a day of M1 candles at a time, going back to 2003 for gold. Naming instruments in dukascopy.pairs turns it on, by the names they use themselves; naming none turns it off, and an install that names only theirs needs no cTrader account at all, since nothing connects until there is something to ask for.

ohlcdb dukascopy -s 2003-05-05

It arrives as a feed of its own - broker Dukascopy, marketplace SWFX - so the same pair from two sources is two pairs, and the sessions, the levels and the page treat both the same way. What their volume counts is what was traded on that marketplace, which is a real quantity with a fraction, where cTrader's is a count of quote updates. Two feeds, two meanings, one column.

They serve that history twice. Their charts read it from an API that answers a day of candles as JSON in a fifth of a second; their archive of files answers the same day in seconds to tens of seconds, and grows slower the longer it is asked. The API is asked first and the files are the reserve, day by day, so a run that would take hours takes minutes and still finishes if the API changes or goes. Rebuilt from the JSON, a day is the same day the file holds, to the last decimal place.

Both turn away a client that does not name itself: a request arriving as Python-urllib is refused, by the API for a minute and a half and by the archive for good. The collector names itself instead, which is all they ask, and what looked for days like heavy throttling was that and nothing else.

A run fetches dukascopy.budget days and leaves the rest to the next run; ohlcdb dukascopy with no budget fetches the lot, which for twenty years - eight and a half thousand days - is under half an hour. A day the market was shut holds nothing, which is not an error and not remembered: asking again next time costs one request, and that is cheaper than a table of absences.

dukascopy.start is where a pair with nothing stored begins; leave it empty and it begins where their history does, which they are asked for. What a price is quoted to - three decimal places for gold, five for a rate quoted to five - they say as well, and a pair keeps the answer it was stored with. The first candle of a run is logged either way, and gold at 4.39 instead of 4390 is hard to miss.

Databento

Databento sell what the exchanges publish. It is the first source here that is paid for, and the first where the shop and the market are not the same company: the prices are the CME's, and they are the ones handing them over.

ohlcdb databento -s 2023-01-01

Naming symbols in databento.symbols turns it on. GC.v.0 is the gold future as a continuous series, rolled onto whichever month is trading most; .n.0 follows open interest and .c.0 the nearest expiry. The series is not adjusted, so there is a price jump at each roll - which matters for anything measured across one and not at all for anything measured inside a session.

It arrives as a feed named after the market, not the vendor: broker CME, market futures, source Databento. Buying the same instrument from somebody else one day then fills this feed rather than standing up a second one with the same prices under another name.

The volume is contracts - a hundred ounces of gold apiece - which is the first volume in this database that counts an amount traded on an exchange rather than one venue's share or a tally of quote updates.

A run buys up to where their history reaches, which is asked for free with the cost and is a few hours ago rather than last midnight: at half past one on a Tuesday afternoon the minutes were there to seven that morning. The end they give carries nanoseconds and is exclusive, so it is floored to the minute, the last one certain to be complete. Where that answer does not arrive the run falls back on midnight, since a metadata call failing is no reason to fail a collection.

What a run would cost is asked before it fetches anything, which is free, and a run that would spend more than databento.spend stops there and says what it would have cost. A day of minutes is about three thousandths of a dollar, so the limit never fires in ordinary use: it is there for the day a date is typed wrong. The download itself goes a month at a time - their meter runs on the data and not on the number of requests, so cutting it up costs nothing and keeps a long history out of memory. The allowance is the run's rather than each symbol's: what one spends the next no longer has.

Contracts

A symbol of the shape GC.FUT is not a series but a product: every contract of it, each one a pair. The shape is what says so - .v.0, .n.0 and .c.0 are continuous series, .FUT a product - because one stype setting cannot describe both, and the two are worth having together: the continuous series covers the years the contracts are not bought for.

What a contract is for is the step. A continuous series is stitched at each roll and not adjusted, so its price jumps where the stitch is - on the Nasdaq about a per cent and a bit, ten or so times a year. It costs nothing to a measurement inside a session and everything to one across days. A contract has no stitch: it begins, it trades, it expires.

Each month is asked for twice. First symbology.resolve, which is free and says which contracts of the product were alive in that month, and then the candles of those alone, bought by instrument id. The list that comes back holds the calendar spreads too - NQM4-NQU4 beside NQM4 - and those are not contracts but the difference of two, so they go by the hyphen in the name. Leaving them out is also most of the bill on gold, where the spread is a real market: every contract of gold from 2010 is $38 against $58 with the spreads, where on the indices the difference is a tenth.

The month is the unit for a second reason. Their symbol carries one digit of the year, so GCZ4 is December 2014 as much as December 2024, and stored under that name the two would share a pair. Inside one month only one of them can be alive, so the century is worked out rather than asked for, and the pair is GCZ24. That holds as long as nothing is listed ten years ahead: gold reaches five and the indices less, crude oil reaches nine.

A contract holds minutes and nothing longer. rollup builds from the pairs the configuration names, and it names the product, so this needs no rule - two hundred contracts with hours nobody draws is what it quietly avoids.

Where a product left off is the newest minute of any of its contracts: the front one trades every minute the market is open, so its frontier is the product's, and a contract listed later is caught by the month it starts trading in. A month that would take the run past what it may spend is not bought at all, and the run says where it stopped: what came before is kept, and the next run carries on from there.

roll_calendar(from_day, to_day) in schema.sql is the other side of this: the contract that traded most in each trading day of each product, which is to say the day the roll happened. The view rolls is the same thing over everything, for queries written by hand.

The stitched series

From that calendar, rebuild_roll_steps() works out the pieces of a continuous series and stitched_bars(parent, from_day, to_day, adjust) returns it: the minutes of whichever contract was the market, lifted onto the level of the newest one. adjust is none for the raw stitch, which is what a bought continuous series is, ratio, which keeps percentages right, or difference, which keeps distances in points right.

Three rules keep the series from being cut where the market was not. The roll goes forward only, so a contract nearer expiry never takes back what a further one took, and an oscillation of a day does not become two junctions. It has to be confirmed, leading two trading days running, and the piece starts on the first of them - the day the market moved, not the day this became sure of it. And the step is measured in the last minute of the overlap: the movement from that close to the next day's is an observed number of the new contract, and the series reproduces it only if the old side is lifted by exactly the distance between them at that moment. The median over the whole overlap day is kept as a guard, since one pair of prices is one stale print away from setting the step for sixteen years; where the two disagree by more than a quarter of a per cent the median is taken and said out loud, and that junction is no longer exact.

The pieces carry both lifts, each cumulative towards the present, so the prices of today are the prices that traded and nothing is stored twice. A new roll changes every price before it, which is why nothing here is materialised: a back-adjusted series is not a table that grows at one end.

A collection works out the pieces of the product it bought, from the first day it touched; ohlcdb roll works out all of them from the start, which is what a history collected before any of this existed needs.

How many decimal places a price has is not asked for. They charge for saying it - the tick of an instrument is in their definition schema, which is bought like any other data - and the prices say it themselves, being whole numbers of that tick. So it is counted off them, from the fewest places that write all but one price in a hundred, and kept at the finest reading a run has seen. It decides how the page writes the number and nothing else: the prices are stored as they arrive.

Longer candles

Dukascopy publish minutes and nothing else, and the level views read H1 and D1: without them a feed of theirs holds twenty years of candles and not one session. rollup says which lengths to build from the M1 ones stored - M5 M15 M30 H1 H4 D1 W1 MN1 by default, empty for none - and a collection does it at the end of its run.

ohlcdb roll [-s DATE] [-p PAIR]

builds them without collecting anything, which is what a history fetched before this existed needs once. Without -s each length carries on from the last candle built, that one made again in case the period was still open when it was; with a date it starts there. Nothing is written that was already right, so a second run in a row writes nothing at all.

Carrying on from the last one is right while history grows forwards. When some of it arrives behind instead - an older start date, a feed backfilled years after the fact - the minutes now reach further back than the candles do, and building goes over the lot rather than stepping over what it cannot see. That happens once, and only then.

A length listed in periods is downloaded from cTrader and left alone: a candle from the broker always beats one worked out here. What is built is not collected, and can be deleted and made again from the minutes - which are the only thing stored that nothing can reproduce.

A day runs from 17:00 to 17:00 in New York, as trading_day() in schema.sql has it and the levels always did, so D1 and up are cut there and not at midnight UTC, which would divide the American session in two. Lengths that fit inside an hour are cut on the clock, where the two agree. H4 breaks at 17:00, 21:00, 01:00 New York time, and the clocks change at 02:00 on a Sunday with the market shut, so no candle is ever built across one.

Levels

levels.sql adds views over the candles with the levels of sessions and periods. Run it after schema.sql, and again after an upgrade that changes it; it rebuilds the views and leaves the sessions as edited:

psql -U ohlcdb -d ohlcdb -f /usr/share/ohlcdb/levels.sql

View	One row per
session_levels	pair, trading day and session
day_sessions	pair and trading day, with the three sessions side by side
period_levels	pair and day, week, month or year

Each row has the open, high, low and close, when the high and the low were made, and the high and low of the one before: the same session the day before, or the previous day, week, month or year.

The views are every pair and every day, which is what a query written by hand usually wants. For one pair, or one stretch of days, there are session_levels_of(pair, from_day, to_day) and period_levels_of(pair, from_day) underneath them: what bounds the work belongs inside, since a WHERE written around a view is applied to the rows it has already computed. Sessions cost about two and a half milliseconds a day, so the difference is between a question and an afternoon.

cTrader hands out closed candles only, so the session or the day under way has none of its own yet: the views rebuild it from the H1 candles and the M1 ones after them, at most a minute old, and its complete column stays false until it is over.

A trading day runs from 17:00 to 17:00 New York time, like cTrader's D1 candles, so the day that opens on Sunday evening is Monday. The sessions are rows of the sessions table, each end a time in its own city: Asia 09:00–18:00 Tokyo, London 08:00–17:00 London, New York 08:00–17:00 New York, Wall Street 09:30–16:00 New York. Periods come from D1; the time of a high or low comes from the finest candles stored, and high_from and low_from say which.

They may overlap, and two of them do: London runs into New York, and Wall Street sits inside it - the hours when the American exchanges are open, which is where that session's volume actually is. Each is measured on its own, so one high can belong to two sessions.

The table is where they are edited: nothing is stored, so a row changed there changes the boxes, the levels, the table under the chart and what analysis.sql measures, over the whole history at once. A session is read from the H1 candles where they cover it and from the minutes where they do not, so an hour it opens or closes in the middle of costs a few M1 rows and is exact all the same.

SELECT day, asia_high, asia_low, london_high, london_low, ny_high, ny_low
FROM day_sessions
WHERE pair = 'XAUUSD.raw'
ORDER BY day DESC
LIMIT 5;

Analysis

analysis.sql measures two things over the stored candles: where the high and the low of a session fall among its minutes - by volume, by how far the minute travelled, and by when in the session it happened - and what becomes of the levels they leave behind. It is not part of the collector and the page never reads it - it is run by hand, reads the candles and writes nothing.

ohlcdb analyse [-b BROKER] [-p PAIR] [-s DATE] [-u DATE] [-v NAME=VALUE]...

Without -b it lists the feeds there are and stops. The dates default to the whole history of the feed. -v passes anything else the file declares - within (5 days for a level to be reached again), after (60 minutes to judge what happened), margin (0.10 of the session's range to count as through) - and giving both a named option and the -v that sets the same thing is an error rather than a silent winner.

It runs through psql, which draws the tables, so this one command wants postgresql-client where the rest of ohlcdb does not. Running the file directly is the same thing with the connection spelled out:

psql -U ohlcdb -d ohlcdb -v broker=Dukascopy -f /usr/share/ohlcdb/analysis.sql

What it found on gold, so that nobody derives it twice:

• Session extremes happen on heavy minutes, and with exchange volume it is not subtle: on CME gold futures over 2010-2024, 65% of session highs fall in the heaviest tenth of their session's minutes and 0.5% in the lightest. Dukascopy's volume, which is one venue's share rather than the market's, shows the same thing at half the strength.
• That volume says nothing about whether the level will hold: the share of revisits that break through is flat across every band of it, in both feeds.
• Nor does it say whether price will come back. It appears to, until the distance is held still, and then it does not.
• What does say is the geometry. How far a high stands above its own session's close sorts the levels from 95% reached again within two hours to 55% a day later, the same ladder on two feeds over different decades - and it costs nothing to compute.
• With the distance held still, volume moves the when rather than the whether: 3.4 hours against 1.5 for the nearest highs, and a sixth off the wait for the furthest.
• The effect is measured against a session's own floor, so it is weakest where that floor is high. Inside Wall Street's hours the extreme minute trades 2.7 times the session average against Asia's 3.2, and volume at equal range separates it least of the four sessions.
• Where in a session the extreme falls has a shape of its own, and an even spread is the wrong yardstick: a walk with nothing in it puts its extreme at one end of the window far more often than in the middle. Held against that shape, 55% of New York's highs fall inside 09:30-16:00 where chance alone gives 52%. The wings hold a quarter of the time and nearly half the extremes because they are the ends of the window.
• A session extreme near its close is a door, not a wall: price goes through it about seven points more often than through a price at the same distance, and having gone through, gives the ground back about six points less often. Both halves replicate on two feeds and both sides, and they hold each other up - an artefact of extremes being where noise gathers would make their breaks false more often, not less.
• A session extreme is a weak level, and it is the level and not the side. A high or a low sitting near where its session closed is gone through about seven points more often than a price at the same distance where nothing happened; in the second band the four estimates - two sides, two feeds, different decades - are 7.2, 8.1, 7.9 and 7.4. Past two fifths of the range from the close, nothing replicates. Near levels are rejected three times in five, which is what makes them look like walls; a price at the same distance is rejected slightly more often still. The rejection is the distance's, and what the level adds pushes the other way.
• Inside a session, how far price travels goes as the square root of what trades: the exponent is 0.52-0.57 across the four windows with an r-squared of 0.79-0.85, over 166,000 tenths of a session. Six times the volume moves the price 2.6 times as far. Across whole days the same fit gives 0.37-0.46 and explains a fifth of it: the hour says more about the movement than the day does.
• What does decide when an extreme happens is which market has just arrived. Asia peaks at Shanghai's open and empties through the Tokyo afternoon to 0.4 of its share; London's extremes are made when the Americans come in, at 2.0; New York and Wall Street are front-loaded and fade all afternoon, the last tenth of New York holding 0.27 of what chance would give it. Highs and lows do it together, so what the profile measures is when the market moves and not which way.

License

GPL-3.0; see LICENSE.