Reclassification rule engine.
Approximates commercial-database item constructions (currently Compustat) by reclassifying standardized line items: stripping D&A out of COGS, folding R&D into SG&A, adding special-item charges back to operating income, netting excise taxes from revenue, etc. The rules are an independent empirical approximation computed from public SEC filings only.
Rules are data, loaded from EDN under resources/edgar/reclass/ and applied per [unit start end] period group on top of the :standardized row set. Reclassified rows are ADDED alongside the original items (never replacing them) and carry provenance: :method :reclassified :rule the rule :id that produced the row :derived-from labels of the operand line items actually used They keep the period fields (:unit :start :end :form) of their operands; per-fact metadata that no longer describes the row (:concept :label :description :taxonomy :accn :fy :fp :frame) is dropped, and :filed is the latest :filed among the operands used.
Rule format: {:id :cogs-ex-da ; unique keyword :target "COGS (Compustat)" ; emitted line-item label :formula [:- "Cost of Revenue" "DP (Compustat)"] :guards [[:lt "DP (Compustat)" "Cost of Revenue"]] :compustat "COGS" ; documentation only :notes "..."} ; documentation only
Formula operators: [:= a] - copy operand a [:+ a b ...] - sum [:- a b ...] - first minus the rest [:neg-sum a ..] - negated sum of the operands present with a non-zero value (nil when none are) Operands are line-item labels; wrap as [:opt label] to treat a missing operand as 0 instead of blocking the rule, and as [:neg label] to use the negated value (for items tagged with the opposite sign convention, e.g. an income-signed "other operating income (expense)" line inside an expense sum). The wrappers nest: [:opt [:neg label]]. Targets of earlier rules can be operands of later ones (applied iteratively, capped at 4 passes).
Guards (all must pass; operands referenced by label): [:lt a b] - value of a strictly less than value of b [:lte a b] - value of a less than or equal to value of b [:ties formula b tol] - formula (same syntax as :formula) evaluates and lies within relative tolerance tol of b's value (an arithmetic tie-out against a reported subtotal) [:concept-not-in a #{c ...}] - a's winning XBRL concept is not in the set (rows without a concept, e.g. derived rows, pass) [:concept-in a #{c ...}] - a exists and its winning XBRL concept is in the set (fails for missing rows and for derived rows, whose concept is nil — use this when the guard must positively identify a reported tag) [:present a] - a line item named a exists in the period (marker rows, e.g. FSDS placement signals) [:absent a] - no line item named a exists in the period
Rules are tried in order; for a given :target the first rule whose operands and guards are satisfied wins (the target is skipped once present).
Reclassification rule engine.
Approximates commercial-database item constructions (currently
Compustat) by reclassifying standardized line items: stripping D&A out of
COGS, folding R&D into SG&A, adding special-item charges back to
operating income, netting excise taxes from revenue, etc. The rules are
an independent empirical approximation computed from public SEC filings
only.
Rules are data, loaded from EDN under resources/edgar/reclass/ and applied
per [unit start end] period group on top of the :standardized row set.
Reclassified rows are ADDED alongside the original items (never replacing
them) and carry provenance:
:method :reclassified
:rule the rule :id that produced the row
:derived-from labels of the operand line items actually used
They keep the period fields (:unit :start :end :form) of their operands;
per-fact metadata that no longer describes the row (:concept :label
:description :taxonomy :accn :fy :fp :frame) is dropped, and :filed is the
latest :filed among the operands used.
Rule format:
{:id :cogs-ex-da ; unique keyword
:target "COGS (Compustat)" ; emitted line-item label
:formula [:- "Cost of Revenue" "DP (Compustat)"]
:guards [[:lt "DP (Compustat)" "Cost of Revenue"]]
:compustat "COGS" ; documentation only
:notes "..."} ; documentation only
Formula operators:
[:= a] - copy operand a
[:+ a b ...] - sum
[:- a b ...] - first minus the rest
[:neg-sum a ..] - negated sum of the operands present with a non-zero
value (nil when none are)
Operands are line-item labels; wrap as [:opt label] to treat a missing
operand as 0 instead of blocking the rule, and as [:neg label] to use the
negated value (for items tagged with the opposite sign convention, e.g.
an income-signed "other operating income (expense)" line inside an
expense sum). The wrappers nest: [:opt [:neg label]]. Targets of earlier
rules can be operands of later ones (applied iteratively, capped at 4
passes).
Guards (all must pass; operands referenced by label):
[:lt a b] - value of a strictly less than value of b
[:lte a b] - value of a less than or equal to value of b
[:ties formula b tol] - formula (same syntax as :formula) evaluates
and lies within relative tolerance tol of
b's value (an arithmetic tie-out against a
reported subtotal)
[:concept-not-in a #{c ...}] - a's winning XBRL concept is not in the set
(rows without a concept, e.g. derived rows,
pass)
[:concept-in a #{c ...}] - a exists and its winning XBRL concept is
in the set (fails for missing rows and for
derived rows, whose concept is nil — use
this when the guard must positively
identify a reported tag)
[:present a] - a line item named a exists in the period
(marker rows, e.g. FSDS placement signals)
[:absent a] - no line item named a exists in the period
Rules are tried in order; for a given :target the first rule whose operands
and guards are satisfied wins (the target is skipped once present).(apply-ruleset rows aux-rows ruleset)Apply a reclassification ruleset to statement rows.
rows - standardized statement rows (seq of maps with :line-item :val :unit :start :end ...) aux-rows - rows from other statements made available as operands per period (e.g. the cash-flow statement's "D&A" for income reclass); they are matched into period groups by [unit start end] and are not included in the output ruleset - EDN ruleset map with a :rules vector (see namespace docstring)
Returns rows plus the reclassified rows. When ruleset is nil or has no rules, returns rows unchanged.
Apply a reclassification ruleset to statement rows.
rows - standardized statement rows (seq of maps with :line-item :val
:unit :start :end ...)
aux-rows - rows from other statements made available as operands per period
(e.g. the cash-flow statement's "D&A" for income reclass);
they are matched into period groups by [unit start end] and are
not included in the output
ruleset - EDN ruleset map with a :rules vector (see namespace docstring)
Returns rows plus the reclassified rows. When ruleset is nil or has no
rules, returns rows unchanged.Compustat reclassification ruleset for the balance sheet. Source: resources/edgar/reclass/compustat-balance.edn
Compustat reclassification ruleset for the balance sheet. Source: resources/edgar/reclass/compustat-balance.edn
Compustat reclassification ruleset for the income statement. Source: resources/edgar/reclass/compustat-income.edn
Compustat reclassification ruleset for the income statement. Source: resources/edgar/reclass/compustat-income.edn
(derived-row by-li used)The row for a formula result: the period fields (:unit :start :end :form ...) of the first operand used, without the per-fact metadata that no longer describes it (:concept :label :description :taxonomy :accn :fy :fp :frame :fsds-tag), and with :filed set to the latest :filed among the operands used. Callers assoc :line-item, :val and provenance. Shared by the reclassification rules and edgar.financials' identities.
The row for a formula result: the period fields (:unit :start :end :form ...) of the first operand used, without the per-fact metadata that no longer describes it (:concept :label :description :taxonomy :accn :fy :fp :frame :fsds-tag), and with :filed set to the latest :filed among the operands used. Callers assoc :line-item, :val and provenance. Shared by the reclassification rules and edgar.financials' identities.
(eval-formula [op & operands] by-li)Evaluate a formula against a {line-item row} map. Returns {:val v :used [labels]} or nil when the formula cannot be evaluated. The single formula evaluator — the standardized-view identity engine (edgar.financials/apply-identities) evaluates through this too, so ops and [:opt] semantics cannot drift between the two layers.
Evaluate a formula against a {line-item row} map. Returns
{:val v :used [labels]} or nil when the formula cannot be evaluated.
The single formula evaluator — the standardized-view identity engine
(edgar.financials/apply-identities) evaluates through this too, so ops
and [:opt] semantics cannot drift between the two layers.(ruleset-for statement)Return the active reclassification ruleset for a statement, for inspection. statement: :income | :balance | :cash-flow Returns the full EDN ruleset map, or nil when the statement has no rules.
Return the active reclassification ruleset for a statement, for inspection. statement: :income | :balance | :cash-flow Returns the full EDN ruleset map, or nil when the statement has no rules.
cljdoc builds & hosts documentation for Clojure/Script libraries
| Ctrl+k | Jump to recent docs |
| ← | Move to previous article |
| → | Move to next article |
| Ctrl+/ | Jump to the search field |