tammany

Tammany is a pure Clojure web feed parser library, compatible with a wide range of format (Atom 1.0, Atom 0.3, RSS 2.0, RSS 0.92, RSS 0.91, RSS 0.90 and RSS 1.0).

Motivation

On the JVM Clojure side, most web feed parsing solutions (including my own clj-rome) depends on the Java Rome library. It is a very full-featured and very battle-tested package that began in the golden age of feeds, and is somewhat showing its (venerable) age. Among other problems, it happens to be very challenging to natively compile with Graal (which I needed). Also, I wanted a solution that could conceivably parse feeds in a mostly compatible way between the multiple modern flavours of Clojure. Thus was tammany born, in an attempt to bring some of the Tammany Rising Sun Lubricator magical smoothness to our old (and dear) ball and chain of open web content syndication.

Limitations

For now, tammany is purely a parsing library, there is NO feed generation tooling. It could be added (probably mostly for the most recent atom and rss formats) with some effort, and I could see the point of it on the babashka side, so open issues if you're interested.

Also, common extensions (like Itunes) are not supported in normalization beacause I have not needed them. Again create Issues or PR if you have the need.

Installation

The library is available on clojars, or of course as a git dep.

{:deps {io.gitlab.canterville/tammany {:mvn/version "0.1.0"}}}

Usage

The only function worth mentioning is process-feed which makes for a quite simple API. It takes an XML String or a Reader and returns a parsed and normalized feed.

$ (require '[io.gitlab.canterville.tammany :as feed])

$ (-> "dev-resources/feeds/sample-atom-1.xml" (io/reader) (feed/process-feed))

{:description
 {:type "html",
  :content "A <em>lot</em> of effort\n    went into making this effortless"},
 :updated #inst "2005-07-31T12:29:29.000-00:00",
 :feed-link "http://example.org/feed.atom",
 :copyright {:type "text", :content "Copyright (c) 2003, Mark Pilgrim"},
 :title {:type "text", :content "dive into mark"},
 :link "http://example.org/",
 :generator "Example Toolkit - 1.0 - http://www.example.com/",
 :entries
 [{:description
   {:type "xhtml",
    :content
    "<:xmlns.http%3A%2F%2Fwww.w3.org%2F1999%2Fxhtml/div>\n        <:xmlns.http%3A%2F%2Fwww.w3.org%2F1999%2Fxhtml/p><:xmlns.http%3A%2F%2Fwww.w3.org%2F1999%2Fxhtml/i>[Update: The Atom draft is finished.]</:xmlns.http%3A%2F%2Fwww.w3.org%2F1999%2Fxhtml/i></:xmlns.http%3A%2F%2Fwww.w3.org%2F1999%2Fxhtml/p>\n      </:xmlns.http%3A%2F%2Fwww.w3.org%2F1999%2Fxhtml/div>"},
   :updated #inst "2005-07-31T12:29:29.000-00:00",
   :title {:type "text", :content "Atom draft-07 snapshot"},
   :link "http://example.org/2005/04/02/atom",
   :authors ["Mark Pilgrim"],
   :enclosures
   ({:href "http://example.org/audio/ph34r_my_podcast.mp3",
     :type "audio/mpeg",
     :length 1337}),
   :published #inst "2003-12-13T12:29:29.000-00:00",
   :guid "tag:example.org,2003:3.2397"}]}

Of particular notice is the metadata on this returned map, which contains namespaced keys with data about the detected feed format and version, and also the raw parsed version of the feed, which can be useful if you need to access something that's not in the necessarily simplified normalized version that hosts all the supported feed formats. It can also contain specifiv vendor extensions.

A spec is available to document the normalized result format.

Compatibility

Clojure

Nothing special, also works with Graal native compilation.

Babashka

Fully compatible, but you need some deps in your bb.edn to support the specs. See the file in the repo for details, but it should look like this:

{:deps {org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha"
                                 :git/sha "644a7fc216e43d5da87b07471b0f87d874107d1a"}
        org.clojure/core.specs.alpha {:mvn/version "0.2.62"}}}

Clojurescript

Pending, The cljs branch mostly works, but getting a portable way to parse xml in js is tricky, and some more work is needed fot it to work outside the browser.

Development

You can run some integration tests on supported platforms with babashka task runner:

bb run run-clj-tests

bb run run-bb-tests

License

Copyright © 2022 Nils Grunwald

Distributed under the Eclipse Public License version 1.0.