Introduction

This report checks if the status of packages on CRAN are due to intermittent failures.

Failures defined as warnings, notes or errors without change on:

R version used (if not stable the same svn snapshot)
The package version (Note that CRAN might modify a package without changing the version)
Their dependencies

Reasons of these failures might be because the packages depend on:

Random generation numbers
Flacky external resources
Other ?

Why is this important?

Because package maintainers of dependencies of that package, R core and CRAN team need to check if the failures are false positives.

This report started because it was suggested as something that the R-repositories working group could help the CRAN team.

Retrieve data

It makes use of tools::CRAN_check_results to retrieve the data.

library("dplyr")
library("tools", include.only = c("package_dependencies", "CRAN_check_results"))
library("flextable", include.only = c("flextable", "autofit"))
# Use a LOCAL environment to check if files can be overwritten on my computer
local_build <- as.logical(Sys.getenv("LOCAL", "FALSE"))
yc <- readRDS("today.RDS")
tc <- CRAN_check_results()
# Added 2023/03/09: sometimes some flavors are reported without status: Omit those
tc <- tc[!is.na(tc$Status),]
if (!interactive() && !local_build) {
  message("Saving today's file.")
  saveRDS(tc, file = "today.RDS")
}

The checks are from multiple flavors release, devel, old release and patched on multiple machines and configurations.

old_flavors <- readRDS("flavors.RDS")
flavors <- unique(tc$Flavor)
# One flavor now present in all is the r-devel-windows-x86_64: skip
flavors <- setdiff(flavors, "r-devel-windows-x86_64")
proto <- data.frame(r_version = character(),
                    os = character(),
                    architecture = character(),
                    other = character())
flavors_df <- strcapture(
  pattern = "r-([[:alnum:]]+)-([[:alnum:]]+)-([[:alnum:]_\\+]+)-?(.*)", 
  x = flavors,
  proto = proto)

# Extract R version used and svn id
h <- "https://www.r-project.org/nosvn/R.check/%s/ggplot2-00check.html"
links <- sprintf(h, flavors)
extract_revision <- function(x) {
  r <- readLines(x, 12)[12]
  version <- strcapture(pattern = "([[:digit:]]\\.[[:digit:]]\\.[[:digit:]])",  
                        x = r, proto = data.frame(version = character()))
  revision <- strcapture(pattern = "(r[[:digit:]]+)",  x = r,
                         proto = data.frame(revision = character()))
  cbind(version, revision)
}
revision <- data.frame(version = character(),
                       revision = character())
for (i in links) {
  revision <- rbind(revision, extract_revision(i))
}

flavors_df <- cbind(flavors = flavors, flavors_df, revision)
if (!interactive() && !local_build) {
  saveRDS(flavors_df, "flavors.RDS")
}

m <- match(tc$Flavor, flavors_df$flavors)
tc_flavors <- cbind(tc, flavors_df[m, ])
flextable(flavors_df) |> 
  autofit()

flavors	r_version	os	architecture	other	version	revision
r-devel-linux-x86_64-debian-clang	devel	linux	x86_64	debian-clang		r89634
r-devel-linux-x86_64-debian-gcc	devel	linux	x86_64	debian-gcc		r89645
r-devel-linux-x86_64-fedora-clang	devel	linux	x86_64	fedora-clang		r89611
r-devel-linux-x86_64-fedora-gcc	devel	linux	x86_64	fedora-gcc		r89578
r-devel-macos-arm64	devel	macos	arm64			r89366
r-patched-linux-x86_64	patched	linux	x86_64		4.5.3	r89645
r-release-linux-x86_64	release	linux	x86_64		4.5.3
r-release-macos-arm64	release	macos	arm64		4.5.2
r-release-macos-x86_64	release	macos	x86_64		4.5.1
r-release-windows-x86_64	release	windows	x86_64		4.5.2	r89426
r-oldrel-macos-arm64	oldrel	macos	arm64		4.4.3
r-oldrel-macos-x86_64	oldrel	macos	x86_64		4.4.1
r-oldrel-windows-x86_64	oldrel	windows	x86_64		4.4.3	r89426

It assumes that the same configuration in one package is used for all. Or in other words that the reports of the configuration (svn revision and version) for the A3 package is the same as for all the other packages.

Warning: This assumption is not always true, but this would require to check each log file on each flavor to verify the R and svn id of each package (which could take too much time and resources).

Overview

Briefly an introduction of how much effort goes into checking

library("ggplot2")
theme_set(theme_minimal())
tc |> 
  filter(!is.na(T_install)) |> 
  ggplot() +
  geom_violin(aes(T_install, Flavor)) +
  scale_x_log10() +
  labs(x = "seconds", title = "Time to install", y = element_blank())

Machines (y axis) vs install time (seconds, x axis), violing plot usually around 10 seconds.

Distribution of install time on each machine.

This means that just to install all the packages on the multiple flavors with a single CPU would take 67 days.

tc |> 
  filter(!is.na(T_check)) |> 
  ggplot() +
  geom_violin(aes(T_check, Flavor), trim = FALSE) +
  scale_x_log10() +
  labs(x = "seconds", title = "Time to check", y = element_blank())

Machines (y axis) vs check time (seconds, x axis), violing plot usually around 100 seconds.

Distribution of checking time on each machine.

This means that to check all the packages on the multiple flavors with a single CPU would take 348 days.

tc |> 
  filter(!is.na(T_total)) |> 
  ggplot() +
  geom_violin(aes(T_total, Flavor)) +
  scale_x_log10() +
  labs(x = "seconds", title = "Time to check and install", y = element_blank())

Machines (y axis) vs total time (seconds, x axis), violing plot usually around 100 seconds.

Distribution of total time on each machine.

This means that to install and check all the packages with a single CPU would take 428 days.

I don’t know the computational cost of 266 days of CPU (every day), but a rough calculation of 2.5 cents per hour means 256.71 dollars daily dedicated to this.

tc |> 
  group_by(Package) |> 
  summarize(Versions = n_distinct(Version)) |> 
  ungroup() |> 
  count(Versions, name = "Packages", sort = TRUE) |> 
  flextable() |> 
  autofit()

Versions	Packages
1	23,063
2	364
3	8

This was surprising, but sometimes checks have multiple versions. Probably when a new version is added and the system don’t catch it for a certain machine.

tc |> 
  group_by(Package) |> 
  summarize(Flavors = n_distinct(Flavor)) |> 
  ungroup() |> 
  count(Flavors, name = "Packages", sort = TRUE) |> 
  flextable() |> 
  autofit()

Flavors	Packages
14	23,275
11	47
9	40
13	21
8	14
3	12
12	12
10	7
5	6
6	1

Similarly, often packages are only tested on few configurations.

Combining both we can have packages with few configurations that have multiple versions being tested.

tc |> 
  group_by(Package) |> 
  summarize(Versions = as.character(n_distinct(Version)),
            Flavors = n_distinct(Flavor)) |> 
  ungroup() |> 
  count(Flavors, Versions, name = "Packages") |> 
  ggplot() +
  geom_tile(aes(Flavors, Versions, fill = log10(Packages))) +
  scale_x_continuous(expand = expansion())

Flavors of machines and versions of packages

Most packages are just tested one version.

But focusing on those that have just one version of the package being tested, most of the machines have packages either OK or with some notes.

man_colors <- c("OK" = "green", "NOTE" = "darkgreen", 
                "WARNING" = "yellow", "ERROR" = "red", "FAILURE" = "black")
tc |> 
  group_by(Package) |> 
  filter(n_distinct(Version) == 1) |> 
  ungroup() |> 
  group_by(Flavor) |> 
  count(Status, name = "packages") |> 
  mutate(perc = packages/sum(packages),
         Status = forcats::fct_relevel(Status, names(man_colors))) |> 
  ggplot() + 
  geom_col(aes(perc, Flavor, fill = Status)) +
  scale_x_continuous(expand = expansion(), labels = scales::percent_format()) +
  scale_fill_manual(values = man_colors) +
  labs(title = "Packages check status", x = element_blank())

On the vertical axis the machine, on the horitzonal axis the packages colored by the status.

Most frequent status is OK or NOTE on all machines.

If we look at the most frequent status report for packages we can see this table:

ts <- tc |> 
  group_by(Package) |> 
  filter(n_distinct(Version) == 1) |> 
  count(Status, name = "flavors") |> 
  ungroup() |> 
  tidyr::pivot_wider(values_from = flavors, names_from = Status, 
                     values_fill = 0) |> 
  count(OK, NOTE, WARNING, ERROR, FAILURE, name = "packages", sort = TRUE)
download.file("https://cran.r-project.org/web/packages/packages.rds", 
              destfile = "packages.RDS") # From the help page
ap <- readRDS("packages.RDS") |> 
  as.data.frame() |> 
  distinct(Package, .keep_all = TRUE)
ap_bioc <- available.packages(repos = BiocManager::repositories()[1:5])
ap_bioc <- cbind(ap_bioc, Additional_repositories = NA)
ap_colm <- intersect(colnames(ap), colnames(ap_bioc))
ap <- rbind(ap[, ap_colm], ap_bioc[, ap_colm])
head(ts) |> 
  flextable() |> 
  autofit()

OK	NOTE	packages
14	0	13,499
12	2	4,408
11	3	1,507
0	14	999
9	5	830
10	4	310

We can see that the most common occurrences are some sort of OK and notes on checks. We can also check the official results on CRAN.

We can see that 0.77%, 0.69%, 0.09%, 0.01%, 0.00% of packages pass all checks without notes.

Now let’s see which of the notes or failures are due to intermittent issues.

Compare

First we need to make sure that we compare the right configurations. They must be the same machine, the same R version and the same svn revision between yesterday and today.

# Compare the previous flavor with today's
m_flavor <- which(flavors_df$flavors %in% old_flavors$flavors)
m_version <- which(flavors_df$version %in% old_flavors$version)
m_revision <- which(flavors_df$revision %in% old_flavors$revision)
tm <- table(c(m_flavor, m_version, m_revision))
compare <- flavors_df$flavors[tm == 3] # Only missing the packages version

All changes

Next, compare the status of the packages if the version of the package is the same.

# Find package on the flavors to compare that haven't changed versions
library("dplyr")
tcc <- filter(tc, Flavor %in% compare) |> 
  select(Flavor, Package, Version, Status) |> 
  arrange(Flavor, Package)
ycc <- filter(yc, Flavor %in% compare) |> 
  select(Flavor, Package, Version, Status) |> 
  arrange(Flavor, Package)

all_checks <- merge(tcc, ycc, by = c("Flavor", "Package"), 
                    suffixes = c(".t", ".y"), all = TRUE) 

possible_packages <- all_checks |> 
  filter(Version.t == Version.y & # Same version
           Status.t != Status.y & # Different status
           !is.na(Status.y) & # No new version or removed package
           !is.na(Status.t)) |> 
  rename(Today = Status.t, Yesterday = Status.y)
possible_packages |> 
  select(Package, Flavor, Today, Yesterday, -Version.t, -Version.y) |> 
  arrange(Package, Flavor) |> 
  flextable() |> 
  autofit()

Package	Flavor	Today	Yesterday
AutoWMM	r-devel-linux-x86_64-debian-clang	FAILURE	OK
BayesGP	r-devel-linux-x86_64-debian-clang	NOTE	OK
BayesGP	r-devel-linux-x86_64-fedora-clang	OK	NOTE
BayesGP	r-devel-linux-x86_64-fedora-gcc	OK	NOTE
BigDataStatMeth	r-devel-linux-x86_64-debian-clang	ERROR	OK
BigVAR	r-devel-linux-x86_64-debian-clang	ERROR	OK
BigVAR	r-oldrel-windows-x86_64	ERROR	OK
CohortCharacteristics	r-devel-linux-x86_64-debian-clang	ERROR	OK
CohortCharacteristics	r-oldrel-windows-x86_64	ERROR	NOTE
EpiNow2	r-oldrel-windows-x86_64	NOTE	WARNING
FRK	r-devel-linux-x86_64-debian-clang	NOTE	OK
FracFixR	r-oldrel-windows-x86_64	OK	FAILURE
GeoAdjust	r-devel-linux-x86_64-debian-clang	NOTE	OK
IP	r-devel-linux-x86_64-debian-clang	NOTE	OK
IP	r-devel-linux-x86_64-fedora-clang	NOTE	OK
IP	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
JFE	r-devel-linux-x86_64-debian-clang	OK	ERROR
JFE	r-oldrel-windows-x86_64	OK	ERROR
LocalCop	r-devel-linux-x86_64-debian-clang	NOTE	OK
MLBC	r-devel-linux-x86_64-debian-clang	NOTE	OK
MLZ	r-devel-linux-x86_64-debian-clang	NOTE	OK
MSUthemes	r-devel-linux-x86_64-debian-clang	ERROR	OK
Matrix	r-devel-linux-x86_64-debian-clang	NOTE	OK
Matrix	r-devel-linux-x86_64-fedora-clang	NOTE	OK
Matrix	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
ODT	r-oldrel-windows-x86_64	ERROR	OK
QCA	r-devel-linux-x86_64-debian-clang	NOTE	OK
QCA	r-devel-linux-x86_64-fedora-clang	NOTE	OK
QCA	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
QFASA	r-devel-linux-x86_64-debian-clang	NOTE	OK
RCPA	r-oldrel-windows-x86_64	ERROR	OK
RCurl	r-devel-linux-x86_64-debian-clang	NOTE	OK
RCurl	r-devel-linux-x86_64-fedora-clang	NOTE	OK
RCurl	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
RMVL	r-devel-linux-x86_64-debian-clang	ERROR	OK
RSiena	r-devel-linux-x86_64-debian-clang	NOTE	OK
RSiena	r-devel-linux-x86_64-fedora-clang	NOTE	OK
RSiena	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
RcppClassicExamples	r-devel-linux-x86_64-debian-clang	ERROR	NOTE
S7	r-devel-linux-x86_64-debian-clang	NOTE	OK
S7	r-devel-linux-x86_64-fedora-clang	NOTE	OK
S7	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
SAMtool	r-devel-linux-x86_64-debian-clang	NOTE	OK
SIGN	r-oldrel-windows-x86_64	ERROR	NOTE
SPAS	r-devel-linux-x86_64-debian-clang	NOTE	OK
SpatialGEV	r-devel-linux-x86_64-debian-clang	NOTE	OK
SuperLearner	r-oldrel-windows-x86_64	OK	FAILURE
WARDEN	r-devel-linux-x86_64-debian-clang	NOTE	OK
XML	r-devel-linux-x86_64-debian-clang	NOTE	OK
XML	r-devel-linux-x86_64-fedora-clang	NOTE	OK
XML	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
adbcdrivermanager	r-devel-linux-x86_64-debian-clang	NOTE	OK
adbcdrivermanager	r-devel-linux-x86_64-fedora-clang	NOTE	OK
adbcdrivermanager	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
agregR	r-devel-linux-x86_64-debian-clang	ERROR	OK
arrangements	r-devel-linux-x86_64-debian-clang	NOTE	OK
arrangements	r-devel-linux-x86_64-fedora-clang	NOTE	OK
arrangements	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
arrow	r-devel-linux-x86_64-debian-clang	NOTE	OK
arrow	r-devel-linux-x86_64-fedora-clang	NOTE	OK
arrow	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
artpack	r-oldrel-windows-x86_64	OK	ERROR
asymLD	r-devel-linux-x86_64-debian-clang	ERROR	NOTE
backports	r-devel-linux-x86_64-debian-clang	NOTE	OK
backports	r-devel-linux-x86_64-fedora-clang	NOTE	OK
backports	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
bage	r-devel-linux-x86_64-debian-clang	NOTE	OK
bbotk	r-devel-linux-x86_64-debian-clang	NOTE	OK
bbotk	r-devel-linux-x86_64-fedora-clang	NOTE	OK
bbotk	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
beezdemand	r-devel-linux-x86_64-debian-clang	NOTE	OK
bit64	r-devel-linux-x86_64-debian-clang	WARNING	OK
bluebike	r-devel-linux-x86_64-debian-clang	NOTE	ERROR
box	r-devel-linux-x86_64-debian-clang	NOTE	OK
box	r-devel-linux-x86_64-fedora-clang	NOTE	OK
box	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
cartographer	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
cartographer	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
clarabel	r-oldrel-windows-x86_64	ERROR	NOTE
cleancall	r-devel-linux-x86_64-debian-clang	NOTE	OK
cleancall	r-devel-linux-x86_64-fedora-clang	NOTE	OK
cleancall	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
cli	r-devel-linux-x86_64-debian-clang	NOTE	OK
cli	r-devel-linux-x86_64-fedora-clang	NOTE	OK
cli	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
clustTMB	r-devel-linux-x86_64-debian-clang	NOTE	OK
coat	r-devel-linux-x86_64-debian-clang	NOTE	OK
collections	r-devel-linux-x86_64-debian-clang	NOTE	OK
collections	r-devel-linux-x86_64-fedora-clang	NOTE	OK
collections	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
crc32c	r-oldrel-windows-x86_64	OK	ERROR
cricketdata	r-devel-linux-x86_64-debian-clang	ERROR	OK
dang	r-oldrel-windows-x86_64	OK	ERROR
dbarts	r-devel-linux-x86_64-debian-clang	NOTE	OK
dbarts	r-devel-linux-x86_64-fedora-clang	NOTE	OK
dbarts	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
deSolve	r-devel-linux-x86_64-debian-clang	NOTE	OK
deSolve	r-devel-linux-x86_64-fedora-clang	NOTE	OK
deSolve	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
depmixS4	r-devel-linux-x86_64-debian-clang	NOTE	OK
dipsaus	r-devel-linux-x86_64-debian-clang	NOTE	OK
dipsaus	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
disaggregation	r-devel-linux-x86_64-debian-clang	NOTE	OK
dlookr	r-devel-linux-x86_64-debian-clang	ERROR	OK
dplyr	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
dplyr	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
dplyr	r-oldrel-windows-x86_64	ERROR	OK
dsem	r-devel-linux-x86_64-debian-clang	NOTE	OK
dsem	r-devel-linux-x86_64-fedora-clang	OK	NOTE
dsem	r-devel-linux-x86_64-fedora-gcc	OK	NOTE
duckplyr	r-oldrel-windows-x86_64	ERROR	OK
duckspatial	r-devel-linux-x86_64-debian-clang	ERROR	OK
duckspatial	r-oldrel-windows-x86_64	ERROR	OK
dynamAedes	r-devel-linux-x86_64-fedora-clang	ERROR	OK
dynamAedes	r-devel-linux-x86_64-fedora-gcc	ERROR	OK
ef	r-devel-linux-x86_64-debian-clang	NOTE	OK
ellipsis	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
ellipsis	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
ellipsis	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
enerscape	r-devel-linux-x86_64-debian-clang	WARNING	OK
epigrowthfit	r-devel-linux-x86_64-debian-clang	NOTE	OK
etwfe	r-devel-linux-x86_64-fedora-clang	ERROR	OK
etwfe	r-devel-linux-x86_64-fedora-gcc	ERROR	OK
fastplyr	r-devel-linux-x86_64-debian-clang	NOTE	OK
fastplyr	r-devel-linux-x86_64-fedora-clang	NOTE	OK
fastplyr	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
filearray	r-devel-linux-x86_64-debian-clang	ERROR	OK
fishmethods	r-devel-linux-x86_64-debian-clang	NOTE	OK
fozziejoin	r-oldrel-windows-x86_64	ERROR	OK
geofi	r-devel-linux-x86_64-debian-clang	OK	ERROR
gghdx	r-devel-linux-x86_64-debian-clang	ERROR	OK
ggtaxplot	r-devel-linux-x86_64-debian-clang	ERROR	OK
ggtaxplot	r-oldrel-windows-x86_64	ERROR	OK
gllvm	r-devel-linux-x86_64-debian-clang	NOTE	OK
glmmTMB	r-devel-linux-x86_64-debian-clang	NOTE	OK
graphonmix	r-devel-linux-x86_64-debian-clang	OK	ERROR
hmmTMB	r-devel-linux-x86_64-debian-clang	NOTE	OK
houba	r-devel-linux-x86_64-debian-clang	ERROR	OK
htetree	r-devel-linux-x86_64-debian-clang	NOTE	OK
htetree	r-devel-linux-x86_64-fedora-clang	NOTE	OK
htetree	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
igraph	r-devel-linux-x86_64-debian-clang	NOTE	OK
igraph	r-devel-linux-x86_64-fedora-clang	NOTE	OK
igraph	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
immundata	r-devel-linux-x86_64-debian-clang	OK	ERROR
lazyeval	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
lazyeval	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
lipidmapsR	r-oldrel-windows-x86_64	ERROR	OK
lmom	r-devel-linux-x86_64-debian-clang	NOTE	OK
lmom	r-devel-linux-x86_64-fedora-clang	NOTE	OK
lmom	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
lobstr	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
lobstr	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
lobstr	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
ltertools	r-oldrel-windows-x86_64	ERROR	OK
magrittr	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
magrittr	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
mda.biber	r-devel-linux-x86_64-debian-clang	OK	ERROR
meme	r-devel-linux-x86_64-debian-clang	ERROR	OK
misha	r-devel-linux-x86_64-debian-clang	NOTE	OK
misha	r-devel-linux-x86_64-fedora-clang	NOTE	OK
misha	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
mmap	r-devel-linux-x86_64-debian-clang	NOTE	OK
mmap	r-devel-linux-x86_64-fedora-clang	NOTE	OK
mmap	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
mmrm	r-devel-linux-x86_64-debian-clang	NOTE	OK
nanonext	r-devel-linux-x86_64-debian-clang	NOTE	OK
nanonext	r-devel-linux-x86_64-fedora-clang	NOTE	OK
nanonext	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
naryn	r-devel-linux-x86_64-debian-clang	NOTE	OK
naryn	r-devel-linux-x86_64-fedora-clang	NOTE	OK
naryn	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
nimble	r-devel-linux-x86_64-debian-clang	NOTE	OK
osdc	r-devel-linux-x86_64-debian-clang	OK	ERROR
oskeyring	r-devel-linux-x86_64-debian-clang	NOTE	OK
oskeyring	r-devel-linux-x86_64-fedora-clang	NOTE	OK
oskeyring	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
otelsdk	r-devel-linux-x86_64-debian-clang	NOTE	OK
otelsdk	r-devel-linux-x86_64-fedora-clang	NOTE	OK
otelsdk	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
pacta.loanbook	r-oldrel-windows-x86_64	OK	ERROR
panelsummary	r-devel-linux-x86_64-fedora-clang	ERROR	NOTE
panelsummary	r-devel-linux-x86_64-fedora-gcc	ERROR	NOTE
pbdMPI	r-devel-linux-x86_64-debian-clang	NOTE	OK
pbdMPI	r-devel-linux-x86_64-fedora-clang	NOTE	OK
pbdMPI	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
phylosem	r-devel-linux-x86_64-debian-clang	NOTE	OK
phylosem	r-devel-linux-x86_64-fedora-clang	OK	NOTE
phylosem	r-devel-linux-x86_64-fedora-gcc	OK	NOTE
pkgdepends	r-devel-linux-x86_64-debian-clang	NOTE	OK
pkgdepends	r-devel-linux-x86_64-fedora-clang	NOTE	OK
pkgdepends	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
planr	r-devel-linux-x86_64-fedora-clang	ERROR	OK
planr	r-devel-linux-x86_64-fedora-gcc	ERROR	OK
precondition	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
precondition	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
processx	r-devel-linux-x86_64-debian-clang	NOTE	OK
processx	r-devel-linux-x86_64-fedora-clang	NOTE	OK
processx	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
ps	r-devel-linux-x86_64-debian-clang	NOTE	OK
ps	r-devel-linux-x86_64-fedora-clang	NOTE	OK
ps	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
purrr	r-devel-linux-x86_64-debian-clang	NOTE	OK
purrr	r-devel-linux-x86_64-fedora-clang	NOTE	OK
purrr	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
qbrms	r-devel-linux-x86_64-debian-clang	NOTE	OK
rATTAINS	r-oldrel-windows-x86_64	ERROR	OK
ravecore	r-devel-linux-x86_64-debian-clang	ERROR	OK
ravepipeline	r-devel-linux-x86_64-debian-clang	ERROR	OK
rbranding	r-devel-linux-x86_64-debian-clang	ERROR	OK
renv	r-devel-linux-x86_64-debian-clang	NOTE	OK
reticulate	r-devel-linux-x86_64-debian-clang	NOTE	OK
reticulate	r-devel-linux-x86_64-fedora-clang	NOTE	OK
reticulate	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
riskmetric	r-oldrel-windows-x86_64	ERROR	OK
robscale	r-devel-linux-x86_64-fedora-clang	ERROR	OK
robscale	r-devel-linux-x86_64-fedora-gcc	ERROR	OK
rpart	r-devel-linux-x86_64-debian-clang	NOTE	OK
rpart	r-devel-linux-x86_64-fedora-clang	NOTE	OK
rpart	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
rprojroot	r-oldrel-windows-x86_64	OK	ERROR
rswipl	r-devel-linux-x86_64-fedora-gcc	ERROR	NOTE
rzmq	r-devel-linux-x86_64-debian-clang	NOTE	OK
rzmq	r-devel-linux-x86_64-fedora-clang	NOTE	OK
rzmq	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
scPOEM	r-oldrel-windows-x86_64	ERROR	OK
sdmTMB	r-devel-linux-x86_64-debian-clang	NOTE	OK
shinyscholar	r-oldrel-windows-x86_64	ERROR	OK
showtext	r-devel-linux-x86_64-debian-clang	ERROR	NOTE
showtext	r-devel-linux-x86_64-fedora-clang	NOTE	OK
showtext	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
softclassval	r-devel-linux-x86_64-debian-clang	NOTE	OK
sovereign	r-devel-linux-x86_64-debian-clang	ERROR	OK
sovereign	r-oldrel-windows-x86_64	ERROR	OK
spaMM	r-devel-linux-x86_64-debian-clang	NOTE	OK
spaMM	r-devel-linux-x86_64-fedora-clang	NOTE	OK
spaMM	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
spatialreg	r-devel-linux-x86_64-debian-clang	NOTE	OK
spatialreg	r-devel-linux-x86_64-fedora-clang	NOTE	OK
spatialreg	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
spatstat.random	r-devel-linux-x86_64-debian-clang	NOTE	OK
spatstat.random	r-devel-linux-x86_64-fedora-clang	NOTE	OK
spatstat.random	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
srcpkgs	r-devel-linux-x86_64-debian-clang	ERROR	OK
srcpkgs	r-oldrel-windows-x86_64	ERROR	OK
ssdtools	r-devel-linux-x86_64-debian-clang	NOTE	OK
stan4bart	r-devel-linux-x86_64-debian-clang	NOTE	OK
stan4bart	r-devel-linux-x86_64-fedora-clang	NOTE	OK
stan4bart	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
stelfi	r-devel-linux-x86_64-debian-clang	NOTE	OK
stochvolTMB	r-devel-linux-x86_64-debian-clang	NOTE	OK
sweep	r-devel-linux-x86_64-debian-clang	ERROR	OK
tablespan	r-oldrel-windows-x86_64	ERROR	OK
tgstat	r-devel-linux-x86_64-debian-clang	NOTE	OK
tgstat	r-devel-linux-x86_64-fedora-clang	NOTE	OK
tgstat	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
this.path	r-devel-linux-x86_64-debian-clang	NOTE	OK
this.path	r-devel-linux-x86_64-fedora-clang	NOTE	OK
this.path	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
tidyquant	r-devel-linux-x86_64-debian-clang	ERROR	OK
tinyVAST	r-devel-linux-x86_64-debian-clang	NOTE	OK
tinyVAST	r-devel-linux-x86_64-fedora-gcc	OK	NOTE
tmbstan	r-devel-linux-x86_64-fedora-clang	NOTE	OK
tmbstan	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
torch	r-devel-linux-x86_64-debian-clang	NOTE	OK
torch	r-devel-linux-x86_64-fedora-clang	NOTE	OK
torch	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
treesitter	r-devel-linux-x86_64-debian-clang	NOTE	OK
treesitter	r-devel-linux-x86_64-fedora-clang	NOTE	OK
treesitter	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
tsdistributions	r-devel-linux-x86_64-fedora-gcc	OK	NOTE
tsissm	r-devel-linux-x86_64-debian-clang	NOTE	OK
ucminf	r-devel-linux-x86_64-debian-clang	NOTE	OK
ucminf	r-devel-linux-x86_64-fedora-clang	NOTE	OK
ucminf	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
unmarked	r-devel-linux-x86_64-debian-clang	NOTE	OK
valr	r-oldrel-windows-x86_64	ERROR	OK
vetr	r-devel-linux-x86_64-debian-clang	NOTE	OK
vetr	r-devel-linux-x86_64-fedora-clang	NOTE	OK
vetr	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
vroom	r-devel-linux-x86_64-debian-clang	NOTE	OK
vroom	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
webfakes	r-devel-linux-x86_64-debian-clang	NOTE	OK
webfakes	r-devel-linux-x86_64-fedora-clang	NOTE	OK
webfakes	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
xLLiM	r-oldrel-windows-x86_64	OK	ERROR
xaringanthemer	r-devel-linux-x86_64-debian-clang	ERROR	OK
xfun	r-oldrel-windows-x86_64	OK	ERROR
yyjsonr	r-devel-linux-x86_64-debian-clang	NOTE	OK
yyjsonr	r-devel-linux-x86_64-fedora-clang	NOTE	OK
yyjsonr	r-devel-linux-x86_64-fedora-gcc	NOTE	OK

If the machine and R versions is the same but the check of the package is different there might be some discrepancy between the dependencies.

# Extract dependencies
dependencies <- package_dependencies(unique(possible_packages$Package),
                                     # Should it check all the recursive dependencies or only direct?
                                     db = ap, # Only considering those dependencies on CRAN and Bioconductor but not any Additional_repositories. 
                                     recursive = TRUE, 
                                     which = c("Depends", "Imports", "LinkingTo", "Suggests"))

# Prepare to compare versions (as they are sorted by everything else we can compare directly)
intermittent_failures <- rep(FALSE, length(dependencies))
names(intermittent_failures) <- names(dependencies)
dep_0 <- lengths(dependencies) == 0
intermittent_failures[dep_0] <- TRUE

If they do not have any recursive dependency on Depends, Imports, LinkingTo and Suggests they might be have some intermittent problems on the packages. These is only on dependencies on CRAN and Bioconductor but not in other additional repositories (There are 187 packages with additional repositories).

If they have some dependencies and those dependencies didn’t change as far as we can tell then there might be some problems with random numbers or connectivity.

for (pkg in names(intermittent_failures[!intermittent_failures])) {
  dep <- dependencies[[pkg]]
  fl <- possible_packages$Flavor[possible_packages$Package == pkg]
  intermittent_failures[pkg] <- all_checks |> 
    filter(Package %in% dep,
           Flavor %in% fl,
           Version.t == Version.y,
           Status.t != Status.y) |> 
    nrow() == 0 # If packages outside || any(!dep %in% rownames(ap)) 
}
packages <- names(intermittent_failures)[intermittent_failures]

We finally show the differences on the status of those without any dependency change on version or status¹:

keep_files <- filter(possible_packages, Package %in% packages) |> 
  merge(y = flavors_df, by.x = "Flavor", by.y = "flavors", all.x = TRUE, all.y = FALSE) |> 
  select(Package, Flavor, Version = Version.t, R_version = r_version, OS = os, 
         architecture, other, version, revision) |> 
  mutate(Date = Sys.time())

if (nrow(keep_files >= 1)) {
  write.csv(keep_files, 
            paste0("cran-failing-", format(Sys.time(), "%Y%m%dT%H%M"), ".csv"),
            row.names = FALSE,
            quote = FALSE,
  )
}

filter(possible_packages, Package %in% packages) |> 
  select(Package, Flavor, Today, Yesterday, -Version.t, -Version.y) |> 
  flextable() |> 
  autofit()

Package	Flavor	Today	Yesterday
backports	r-devel-linux-x86_64-debian-clang	NOTE	OK
lmom	r-devel-linux-x86_64-debian-clang	NOTE	OK
mmap	r-devel-linux-x86_64-debian-clang	NOTE	OK
pbdMPI	r-devel-linux-x86_64-debian-clang	NOTE	OK
RMVL	r-devel-linux-x86_64-debian-clang	ERROR	OK
rzmq	r-devel-linux-x86_64-debian-clang	NOTE	OK
ucminf	r-devel-linux-x86_64-debian-clang	NOTE	OK
backports	r-devel-linux-x86_64-fedora-clang	NOTE	OK
lmom	r-devel-linux-x86_64-fedora-clang	NOTE	OK
mmap	r-devel-linux-x86_64-fedora-clang	NOTE	OK
pbdMPI	r-devel-linux-x86_64-fedora-clang	NOTE	OK
rzmq	r-devel-linux-x86_64-fedora-clang	NOTE	OK
ucminf	r-devel-linux-x86_64-fedora-clang	NOTE	OK
backports	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
lmom	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
mmap	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
pbdMPI	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
rzmq	r-devel-linux-x86_64-fedora-gcc	NOTE	OK
ucminf	r-devel-linux-x86_64-fedora-gcc	NOTE	OK

Conclusion

cat("There are no packages detected with differences between yesterday and today attributable to intermittent failures.\n")
knitr::knit_exit()

cat("This suggests that these packages might have some problems with random numbers or connectivity:\n\n")

This suggests that these packages might have some problems with random numbers or connectivity:

if (any(dep_0)) {
  cat("\n## Packages with dependencies\n\n")
  cat(paste0(" - ", sort(intersect(packages, 
                                   names(dependencies)[dep_0])), "\n"), sep = "")
  cat("\n## Packages without dependencies\n\n")
  cat(paste0(" - ", sort(intersect(packages,
                                   names(dependencies)[!dep_0])), "\n"), sep = "")
  
} else {
  cat(paste0(" - ", sort(packages), "\n"), sep = "")
}

Packages with dependencies

backports
mmap
RMVL
rzmq

Packages without dependencies

lmom
pbdMPI
ucminf

I think a new version might not propagate to check other packages until 24 hours later as checks might have already started for that day.↩︎

Packages intermittent failtures

Lluís Revilla Sancho

2026-03-18