Gnus-style scoring for Elfeed.
This manual corresponds to elfeed-score version 0.7.10.
elfeed-score brings Gnus-style scoring to elfeed. If you’re not sure what that means, you’re in the right place. If you know exactly what that means and you just want the technical details, you are also in the right place.
If you are new to elfeed, elfeed-score, or Emacs altogether... read on. This manual is intended to be read front-to-back by new users.
If you are looking for technical details: for a list of elfeed-score functions, refer to the Function Index. For a list of elfeed-score variables, see the Variable Index. For detailed documentation on rule types, see Rule Types.
For information on obtaining & installing elfeed-score, see the project README. For information on building & hacking on elfeed-score, see the Wiki.
elfeed is an extensible RSS & Atom reader for Emacs. By default, it will display entries in reverse chronological order. This package defines a bit of metadata for each of your elfeed entries: a "score". A score is an integer (negative or positive), and higher scores denote entries of greater interest to you. This package also (optionally) installs a new sort function, so that Elfeed will display entries with higher scores before entries with lower scores (entries with the same scores will still be sorted in reverse chronological order). It also provides an entry display function for the search buffer that displays each entry’s score, should you choose to install it.
While you can manually assign a score to an entry (see Manual Scoring), you will likely find it more convenient to create rules for
scoring that will be automatically applied to each new entry every
time you update Elfeed. You can score against title, feed, content &
authors by defining strings that will be matched against those
attributes by substring, regexp or whole-word match. You can score
against the feed. You can also score against the presence or absence
of tags. Most kinds of rules can be scoped by Elfeed entry tags, so
that a rule will only be applied if an entry has certain tags (or does
not have certain tags). Many kinds of rules may also be scoped by
feed, so that a rule will only be applied to entries from certain
feeds (or not from certain feeds). Each rule defines an integral
value, and the rules are applied in order of definition. The new
entry’s score begins at elfeed-score-scoring-default-score
, and
is adjusted by the value defined by each matching scoring rule.
The default score for an elfeed entry.
For instance, here’s a subset of my scoring file at the moment:
;;; Elfeed score file -*- lisp -*- (("title" (:text "OPEN THREAD" :value -1000 :type S) (:text "raymond c\\(hen\\)?" :value 250 :type r) :tags (t .(@dev))) ("content" (:text "type erasure" :value 500 :type s)) ("title-or-content" (:text "california" 150 100 :type s) (:text "china" 150 100 :type w)) ("feed" (:text "Essays in Idleness" :value 250 :type S :attr t) (:text "Irreal" :value 250 :type S :attr t) (:text "Julia Evans" :value 100 :type :type s :attr t) (:text "National Weather Service" :value 400 :type S :attr t) (:text "emacs-news – sacha chua" :value 350 :type S :attr t)) ("authors" (:text "Jim Geraghty" :value 500 :type s)) ("tag" (:tags (t . reddit-question) :value 750)) (mark -2500))
Like Gnus scoring, this may look like Lisp code, but it is not directly evaluated. It will be read by the Lisp reader, so it must at least be a valid Lisp s-expression.
For details of how to form s-expressions elfeed-score will understand, see below.
Scores may also be set on entries manually:
Set the score of one or more Elfeed entries to score (given as a prefix argument).
Since these scores are presumably significant to the user who took the trouble to set them, such scores will not be overwritten by other scoring operations (such as the new entry hook which elfeed-score installs). Since this behavior was implemented only recently, it is controlled by a configuration variable:
This customization variable, if true, will make manual scores “sticky” in that they will not be overwritten by other scoring operations (they can, however, be overwritten by a subsequent manual scoring operation). If nil, manual scoring is treated no differently than any other scoring operation.
elfeed-score maintains statistics on when & how frequently each rule matches an entry (on which more below). In earlier versions of the package, each rule’s stats were maintained in the rule structure itself & were persisted along with each rule to the score file.
This turned out to be ill-conceived, leading to split-brain when the user edited their score file with un-written stats still in memory.
Today, elfeed-score maintains a separate datastructure holding this
information & automatically writes it to disk in a separate file. The
location of the stats file is defined in
elfeed-score-rule-stats-file
.
The location at which scoring rule stats are maintained.
The default value is elfeed.stats in
user-emacs-directory
, which see user-emacs-directory in Emacs Lisp.
Since this file is meant to be maintained automatically by elfeed-score, this manual won’t go over the file format.
Once you’ve setup your score file, and assuming you’ve installed elfeed-score, begin or switch to your Emacs session and say:
(require 'elfeed-score)
Just loading the library will *not* modify elfeed; you need to explicitly enable the package for that:
(elfeed-score-enable)
This will install the new sort function & new entry hook, as well as
read your score & stats files. It will also install an elfeed update
hook; this is to detect when an update has completed so that we can
write-out the new rule stats to disk. NB. elfeed-score-enable
is autoloaded, so if you’ve installed this package in the usual way,
you should be able to just invoke the function & have the package
loaded & enabled automatically.
Install the new sort function, the new entry hook & the update hook. Read the score & stats files. With prefix arg do not install the custom sort function.
Some elfeed users have already customized
elfeed-search-sort-function
and may not wish to have
elfeed-score install a new one. elfeed-score-enable
takes a
prefix argument: if present, it will install the new entry hook &
commence scoring, but will not install the new sort
function. Such users may refer to elfeed-score-sort
if they
would like to incorporate scoring into their sort functions.
elfeed-search-sort-function
-compliant function that will
sort elfeed entries by score first, then timestamp.
The package defines a keymap, but does not bind it to any key. I like to set it to the = key:
(define-key elfeed-search-mode-map "=" elfeed-score-map)
Now, only when you’re in the elfeed-score search buffer, you can access elfeed-score commands on the = key.
elfeed allows us to customize how entries are displayed in the elfeed
search buffer through the variable
elfeed-search-print-entry-function
. Because users have
customized this, elfeed-score does not alter this variable directly
Rather, it offers a replacement implementation,
elfeed-score-print-entry
, should you wish to use it.
Prefix each entry with its score when displayed in the elfeed search buffer.
Install it like so:
(setq elfeed-search-print-entry-function #'elfeed-score-print-entry)
While this is not turned on by elfeed-score-enable
,
elfeed-score-unload
will remove it, if it’s there.
Unload elfeed-score.
Even if you’ve customized the elfeed print function, if you would like
to incorporate the score into your version, you can use the
elfeed-score-print-entry
implementation for inspiration.
The rules for scoring are written down in the score file, a plain-text
file containing a single Lisp form. The location of the score file is
defined in elfeed-score-serde-score-file
.
The location at which scoring rules are maintained.
The default value is elfeed.score in
user-emacs-directory
, which see user-emacs-directory in Emacs Lisp.
We’ll go over the format in more detail below.
It is important to note that the score file is authored & maintained by the user: elfeed-score will only read from it, not write to it. Prior versions of elfeed-score kept assorted statistics about each rule (how many times it matched, for instance) in the rule structures themselves, and would periodically write them out to the score file. This led to situations where users would lose edits to their score file when elfeed-score blindly wrote out its in-memory state. Beginning with elfeed-score 0.7.10, those statistics are tracked & persisted separately, which will hopefully put an end to this problem.
That said, if you update your rules by hand, you need to tell
elfeed-score to re-load it before (re-)scoring any elfeed entries. You
can do this by invoking elfeed-score-load-score-file
(=
l):
Load the score file into the current session.
The score file is plain-text file containing a single Lisp form. Under
the hood, elfeed-score opens your file, inserts its contents into a
temporary buffer, and calls read-from-string
(see read-from-string in Emacs Lisp) on
the contents.
The form is a list of lists; each sub-list begins with a symbol or a string identifying that sub-list’s nature & purpose. The sub-list identifiers elfeed-score recognizes are:
version
The sublist named by the symbol version
contains
the version of the score file format. You do not need to include this
in your initial score file; it is written out automatically by
elfeed-score. elfeed-score maintains backward compatibiliy in that
score files in older format versions are still recognized. The most
recent format version will always be used when the score file is
updated, however. Also, an older elfeed-score will reject a more
recent score file.
When elfeed-score loads a score file that uses an archaic version
(i.e. a version less than elfeed-score-serde-current-format
) it
will announce the fact that your score file will be updated and make a
backup of your current score file in %s.~%d~
where %s
is
replaced by elfeed-score-serde-score-file
and %d
is
replaced by your score file’s current version (so that you will end up
with a copy of the last score file in each format over time).
The version of the score file format employed by elfeed-score.
It will also immediately re-write the score file in the current format.
mark
This sublist contains an integer; if an entry’s final score is below
this value, the entry will be marked as “read”. Details below in
mark Rule.
Until elfeed-score 0.7, all rules were serialized to flat lists, with each attribute in a known position in the list (for instance, title rules wrote down their match text in position 0, the match value in position 1, and so forth, while the other rule types each had their own format).
As time went on, the number of attributes for any given rule type grew. Using flat lists as a serialization format became untenable for two reasons:
i
was for this & slot j
was for that became increasingly burdensome for the package author,
let alone contributors & users.
i
is for this thing, but provides the thing that goes in slot
i + 1
, then when elfeed-score calls (nth in i)
it will
get something back, but will not discover that it’s the wrong
sort of thing for some time, which makes it hard to craft a usable
error message; worse, elfeed-score may never discover it & instead
fail silently.
Beginning with build 0.7.3 (format version 6), elfeed-score now uses property lists as its serialization format. This will address these two issues, at the cost of making authoring rules more prolix: instead of saying:
("match text" 100...)
one now has to say:
(:text "match text" :value 100...)
Given that fields set to their default values do not have to be specified, this will hopefully not be too inconvenient, and will enable better error messages on reading score files.
“title rules” are rules that are matched against each entry’s title. You may match by substring, regexp or whole word. The match may be case sensitive or case-insensitive. Of course, the rule needs to specify the amount to be added to an entry’s score on successful match.
In the score file (see The Score File), title rules are represented by a property list of up to seven attributes:
:text
: The match text.
:value
: The match value.
This is an integer specifying the amount by which the entry’s score
should be adjusted, should the text match
:type
: The match type.
This is a symbol, and must be one of s
,
S
, r
, R
, w
or W
for
case-insensitive substring match, case-sensitive substring match,
case-insensitive regexp match or case-sensitive regexp match, and
case-insensitive or case-sensitive whole word match, respectively.
Note that regular expressions use Emacs Lisp regular expression syntax (see Regular Expressions in Emacs Lisp).
Whole word matching just feeds the match text to
word-search-regexp
(see String
Search in Emacs Lisp) before doing a regexp search.
:tags
: Tag scoping rules.
You may wish to apply this rule only to entries that have certain tags (or that do not have certain tags). See Scoping Rules by Tags for how to do that.
This property is optional.
:feeds
: Feed scoping rules.
You may wish to apply this rule only to entries that came from certain feeds (or did not come from certain feeds). See Scoping Rules by Feed for how to do that.
This property is optional.
So, for instance, the following rule:
("title" (:text "rust.*who.s hiring" :value 1500 :type r))
would match each entry’s title against the regular expression “rust.*who.s hiring” (without regard to case) and adds 1500 to the score of any entry that matches.
Note that it takes advantage of the fact that only the :text
,
:value
& :type
properties are required.
“content rules” are very similar to title Rules except that they match against an entry’s content rather than its title. For that reason, this section will be very brief– refer to title Rules for the details of each element.
In the score file (see The Score File), content rules are represented by seven properties:
:text
The match text
:value
The match value
:type
The match type as per above.
:tags
Tag scoping rules
:feeds
Feed scoping rules
As above, only the first three elements are required.
When searching for text in entries on which to score, it can be convenient to check both title and content. Rather than repeating each rule in both the “title” and “content” elements of the score file, you can use “title-or-content” rules. These rules consist of eight elements, not seven:
:text
The match text, as with title & content rules
:title-value
The title match value
The value by which an entry’s score will be adjusted when the title matches. This allows different scoring values for title & content matches, on the assumption that a title match would be considered more significant.
:content-value
The content match value
The value by which an entry’s score will be adjusted when the content matches.
:type
The match type as per above.
:tags
Tag scoping rules
:feeds
Feed scoping rules
Note that both entry attributes are checked, so both score values have the potential to added to any given entry.
For instance, the following rule:
("title-or-content" (:text "california" :title-value 150 :content-value 100 :type s))
would add 150 to any element who’s title contains the string “california”, as well as 100 if its content also contains that text.
You can adjust an entry’s score on the basis of the feed that produced it with a “feed” rule. For instance I use this to prioritize certain sources. A feed rule is defined by the following seven elements:
:text
: Match text
The text to be matched against the feed title, URL or author
:value
: Match value
The value by which an entry’s score will be adjusted should there be a
match
:type
: Match type
The match type as per above.
:attr
: Feed attribute
One of the symbols t
, u
or a
for title, URL, or
author respectively; this determines the attribute of the feed against
which this rule’s text will be matched.
:tags
: Tag scoping rules
You may wish to apply this rule only to entries that have certain tags
(or that do not have certain tags). See below for how to do that.
For example, this rule:
("feed" (:text "National Weather Service" :value 400 :type S :attr t))
will check the feed title for each entry against the string “National Weather Service” (case will count). Should there be a match, the corresponding entry will have 400 added to its score.
Some feeds are not produced by a single author. A feed that contains academic papers, for instance, may tag each entry with the authors of the paper to which that entry corresponds. You can score against an entry’s authors with an authors rule.
:text
: The match text
For each entry, if the “authors” meta-data is present, the authors’
names will be concatenated (separated by “, “) and matched against
this text for scoring purposes.
:value
: The match value
This is an integer specifying the amount by which the entry’s score
should be adjusted, should the text match
:type
: The match type
:tags
: Tag scoping rules
You may wish to apply this rule only to entries that have certain tags (or that do not have certain tags). See below for how to do that.
:feeds
: Feed scoping rules
You may wish to apply this rule only to entries that came from certain feeds (or did not come from certain feeds). See below for how to do that.
elfeed allows one to automatically tag new entries as they are discovered. It can therefore be convenient to group entries that come from many different feeds but share a tag or tags. That is the purpose of “tag” rules.
“tag” rules are defined by four properties:
:tags
: Tags
The tags whose presence or absence will trigger this rule. They are specified as a cons cell (see cons cell in Emacs Lisp) of the form:
(switch . tags)
switch
is either t
or nil
and tags
is
either a tag (i.e. a symbol) or a list of tags. If switch
is
true, this rule will apply to any entry tagged with one or more tags
listed in tags
. Conversely, if switch
is false, the rule
will apply to entries who posses none of the tags in
tags
.
:value
: Match value
The amount by which an entry’s score shall be adjusted should this
rule match.
“link rules” are rules that are matched against each entry’s Link attribute. They are similar to title Rules excepting the entry attribute to be matched. For that reason, this section will be brief– refer to title Rules for the details of each element.
In the score file (see The Score File), content rules are represented by seven properties:
:text
The match text
:value
The match value
:type
The match type as per above.
:tags
Tag scoping rules
:feeds
Feed scoping rules
As above, only the first three elements are required.
Until now, all the rules have been used to determine an entry’s score. The “adjust-tags” rule (and the “mark” rule below) act on entries after their scores have been determined.
The adust-tags rule was inspired by John Kitchin’s article Scoring Elfeed Articles. He computes a score and adds one or two tags to entries whose score is sufficiently high. It always bothered me that elfeed-score couldn’t do that, so in build 0.4.3, I added this rule type. These will add or remove tags based on whether the entry’s score is above or below a given threshold.
adjust-tags rules are given by four properties:
:threshold
: The threshold at which the rule shall apply
This is defined by a cons cell (see cons cell in Emacs Lisp) of the form:
(switch . threshold)
switch
may be t
or nil
and threshold
is
the threshold against which each entry’s score shall be compared. If
switch
is t
, the rule applies if the score is greater
than or equal to threshold
; if switch
is nil
the rule
applies if score is less than or equal to threshold
.
:tags
: The tags to be added or removed
This is also given by a cons cell
(switch . tags)
If switch
is t
& the rule applies, tags
(either a
single tag or a list of tags) will be added to the entry; if
switch
is nil
, they will be removed
For example, the following rules:
(("adjust-tags" (:threshold (t . 1000) :tags (t . a)) (:threshold (nil . -1000) :tags (nil . b)))
will add the tag 'a
to all entries whose score is 1000 or more,
and remove tag 'b
from all entries whose score is -1000 or
less.
Like the “adjust-tag” rule (see above,
the mark rule operates on an entry after scoring has completed. If
present, it simply specifies an integer; any entry whose score is
below this number will be marked as read (in that it will have the
'unread
tag removed from it). Since bookmarked elfeed searches
typically specify the 'unread
tag as a criterion, this
essentially means “if an entry’s score is below this number, don’t
even show it to me.”
Unlike the other rules, there may only be one mark rule.
Example:
(mark -2500)
You may find that you only want some rules to not be applied to every entry, but only in certain contexts. Many of the rules described above can be scoped to apply to only entries with certain tags, or to only entries from certain feeds.
“title”, “content”, “title-or-content”, “authors” & “feed” rules can be scoped by tag (see Types of Rules). In the description of each of these type of rules above, you will note a slot reserved for “tag scoping rules”. In each case these rules are expressed as a cons cell of the form:
(boolean . (tag...))
The car is a boolean, and the cdr is a list of tags. If the former is
t
, the rule will only be applied if the entry has at least one
of the tags listed. If the boolean value is nil
, the rule will
only apply if the entry has *none* of the tags listed.
For instance, this is an entry in my score file at the time of this writing:
("title-or-content (:text "workspace" :title-value 150 :content-value 75 :type s :date 1611937077.6047099 :tags (t linux) :hits 48)
It performs a case-insensitive substring match against entry title and content, adding 150 & 75 points to an entry’s score on match, respectively. It matched most recently at Friday, January 29, 2021 8:17:57.604 AM PST, and has matched 48 times in total.
Of particular interest here is the fact that it will only be applied
to entries with the tag 'linux
, since “workspace” is fairly
generic term, and I am only specific interested in the term as it
applies to Linux window managers.
“title”, “content”, “title-or-content” & “authors” rules can also be scoped by feed; that is, you can arrange to have such rules only apply to entries from certain feeds (or apply only to entries not from certain feeds) (see Types of Rules).
Each of these rule types reserves a slot in their definiotn for “feed scope rules”. Such fules are expressed as a cons cell:
(boolean . ((attr match-type matex-text)...))
The car is a boolean, and the cdr is a list of feed selectors. If the
former is t
, the rule will only be applied if the entry’s feed
matches at least one of the selectors listed. If the boolean value is
nil
, the rule will only apply if the entry’s feed matches
none of the selectors listed.
Each selector is itself a three-tuple consisting of:
t
, u
or a
for title, URL, or
author respectively; this determines the attribute of the feed against
which this rule’s text will be matched.
Example:
("title" (:text "foo" :value 100 :type s :feeds (t . ((t S "Bar")))))
This defines a title rule See title Rules that checks for a substring matching “foo” in the entry title and adds 100 points to its score on match. However, it will only match entries originating from feeds whose title contains the string “Bar” (case-sensitive).
This is a small subset of my current score file at the time of this writing. Working score files tend to grow much larger than this (see Rule Maintenance below), but it is a complete example.
;;; Elfeed score file -*- lisp -*- ((version 8) ("title" (:text "rust.*who.s hiring" :value 1500 :type r) (:text "Microsoft Security Response Center" :value 1500 :type s)) ("content" (:text "/u/I_am_dom_" :value 250 :type s :tags (t @china))) ("title-or-content" (:text "california" :title-value 150 :content-value 100 :type s) (:text "melpa" :title-value 250 :content-value 150 :type s :tags (t @dev @emacs)) (:text "workspace" :title-value 150 :content-value 75 :type s :tags (t linux))) ("tag" (:tags (t . reddit-question) :value 750)) ("authors" (:text "Jim Geraghty" :value 500 :type s)) ("feed" (:text "Hacker News" :value 150 :type S :attr t)) (mark -2500) ("adjust-tags"))
Once you’ve setup your score file (see The Score File), and
started elfeed-score (see Starting elfeed-score), any new
entries will be scored automatically, but the entries already in your
database have not yet been scored. Scoring is idempotent in that
scoring an entry more than once will always result in it having the
same score assigned. This means you can load an Elfeed search, and
then, in the Elfeed search buffer *elfeed-search*
, score all
results with elfeed-score-score-search
, or = v. When the
command completes, the view will be re-sorted by score. Your stats
file will also have been updated on disk (to record the last time that
each rule matched).
Score the current set of search results.
You can configure score logging by setting the variable
elfeed-score-log-level
.
Level at which ‘elfeed-score’ shall log; may be one of ’debug, ’info, ’warn, or ’error.
By default it will be 'warn
, which will produce very little
output. To trouble-shoot a balky rule, type
(setq elfeed-score-log-log-level 'debug)
re-score your current view (= s), and switch to buffer
*elfeed-score*
.
You can also trouble-shoot rules using the “explain” feature below.
If you’ve got an entry that’s not being scored in the manner you
expect, you can ask elfeed-score to explain itself by selecting the
offending entry & invoking elfeed-score-scoring-explain-entry
(= x):
Explain an Elfeed ENTRY.
This function will apply all scoring rules to an entry, but will not change anything (e.g. update the entry’s meta-data, or the last-matched timestamp); instead, it will provide a human-readable description of what would happen if the entry were to be scored, presumably for purposes of debugging or understanding of scoring rules.
As time goes by, you may find your score file growing considerably in size & complexity. elfeed-score offers a few reporting functions for looking at the totality of your rules in a few ways:
Display all scoring rules in descending order of last match.
CATEGORY may be used to narrow the scope of rules displayed. If nil, display all rules. If one of the following symbols, display only that category of rules:
:title :feed :content :title-or-content :authors :tag :adjust-tags
Finally, CATEGORY may be a list of symbols in the preceding list, in which case the union of the corresponding rule categories will be displayed.
Display all scoring rules in descending order of match hits.
CATEGORY may be used to narrow the scope of rules displayed. If nil, display all rules. If one of the following symbols, display only that category of rules:
:title :feed :content :title-or-content :authors :tag :adjust-tags
Finally, CATEGORY may be a list of symbols in the preceding list, in which case the union of the corresponding rule categories will be displayed.
I’ve sketched some thoughts on elfeed-score’s future in the README. The focus now is getting it to a place where I’d feel comfortable calling it “1.0”.
Bugs & feature requests are welcome in the Issues section of the project.
If you’d like to hack on elfeed-score, I’ve started a Wiki for that and am always happy to discuss PRs.
Finally, you can just reach out directly at sp1ff@pobox.com, or in my public inbox.
Jump to: | A C E F L M N R S T U |
---|
Jump to: | A C E F L M N R S T U |
---|
Jump to: | C E |
---|
Jump to: | C E |
---|