Introduction

This report checks if the status of packages on CRAN are due to intermittent failures.

Failures defined as warnings, notes or errors without change on:

R version used (if not stable the same svn snapshot)
The package version (Note that CRAN might modify a package without changing the version)
Their dependencies

Reasons of these failures might be because the packages depend on:

Random generation numbers
Flacky external resources
Other ?

Why is this important?

Because package maintainers of dependencies of that package, R core and CRAN team need to check if the failures are false positives.

This report started because it was suggested as something that the R-repositories working group could help the CRAN team.

Retrieve data

It makes use of tools::CRAN_check_results to retrieve the data.

library("dplyr")
library("tools", include.only = c("package_dependencies", "CRAN_check_results"))
library("flextable", include.only = c("flextable", "autofit"))
# Use a LOCAL environment to check if files can be overwritten on my computer
local_build <- as.logical(Sys.getenv("LOCAL", "FALSE"))
yc <- readRDS("today.RDS")
tc <- CRAN_check_results()
# Added 2023/03/09: sometimes some flavors are reported without status: Omit those
tc <- tc[!is.na(tc$Status),]
if (!interactive() && !local_build) {
  message("Saving today's file.")
  saveRDS(tc, file = "today.RDS")
}

The checks are from multiple flavors release, devel, old release and patched on multiple machines and configurations.

old_flavors <- readRDS("flavors.RDS")
flavors <- unique(tc$Flavor)
# One flavor now present in all is the r-devel-windows-x86_64: skip
flavors <- setdiff(flavors, "r-devel-windows-x86_64")
proto <- data.frame(r_version = character(),
                    os = character(),
                    architecture = character(),
                    other = character())
flavors_df <- strcapture(
  pattern = "r-([[:alnum:]]+)-([[:alnum:]]+)-([[:alnum:]_\\+]+)-?(.*)", 
  x = flavors,
  proto = proto)

# Extract R version used and svn id
h <- "https://www.r-project.org/nosvn/R.check/%s/ggplot2-00check.html"
links <- sprintf(h, flavors)
extract_revision <- function(x) {
  r <- readLines(x, 12)[12]
  version <- strcapture(pattern = "([[:digit:]]\\.[[:digit:]]\\.[[:digit:]])",  
                        x = r, proto = data.frame(version = character()))
  revision <- strcapture(pattern = "(r[[:digit:]]+)",  x = r,
                         proto = data.frame(revision = character()))
  cbind(version, revision)
}
revision <- data.frame(version = character(),
                       revision = character())
for (i in links) {
  revision <- rbind(revision, extract_revision(i))
}

flavors_df <- cbind(flavors = flavors, flavors_df, revision)
if (!interactive() && !local_build) {
  saveRDS(flavors_df, "flavors.RDS")
}

m <- match(tc$Flavor, flavors_df$flavors)
tc_flavors <- cbind(tc, flavors_df[m, ])
flextable(flavors_df) |> 
  autofit()

flavors	r_version	os	architecture	other	version	revision
r-devel-linux-x86_64-debian-clang	devel	linux	x86_64	debian-clang		r89649
r-devel-linux-x86_64-debian-gcc	devel	linux	x86_64	debian-gcc		r89667
r-devel-linux-x86_64-fedora-clang	devel	linux	x86_64	fedora-clang		r89611
r-devel-linux-x86_64-fedora-gcc	devel	linux	x86_64	fedora-gcc		r89578
r-devel-macos-arm64	devel	macos	arm64			r89366
r-patched-linux-x86_64	patched	linux	x86_64		4.5.3	r89645
r-release-linux-x86_64	release	linux	x86_64		4.5.3
r-release-macos-arm64	release	macos	arm64		4.5.2
r-release-macos-x86_64	release	macos	x86_64		4.5.1
r-release-windows-x86_64	release	windows	x86_64		4.5.3
r-oldrel-macos-arm64	oldrel	macos	arm64		4.4.3
r-oldrel-macos-x86_64	oldrel	macos	x86_64		4.4.1
r-oldrel-windows-x86_64	oldrel	windows	x86_64		4.4.3	r89426

It assumes that the same configuration in one package is used for all. Or in other words that the reports of the configuration (svn revision and version) for the A3 package is the same as for all the other packages.

Warning: This assumption is not always true, but this would require to check each log file on each flavor to verify the R and svn id of each package (which could take too much time and resources).

Overview

Briefly an introduction of how much effort goes into checking

library("ggplot2")
theme_set(theme_minimal())
tc |> 
  filter(!is.na(T_install)) |> 
  ggplot() +
  geom_violin(aes(T_install, Flavor)) +
  scale_x_log10() +
  labs(x = "seconds", title = "Time to install", y = element_blank())

Machines (y axis) vs install time (seconds, x axis), violing plot usually around 10 seconds.

Distribution of install time on each machine.

This means that just to install all the packages on the multiple flavors with a single CPU would take 67 days.

tc |> 
  filter(!is.na(T_check)) |> 
  ggplot() +
  geom_violin(aes(T_check, Flavor), trim = FALSE) +
  scale_x_log10() +
  labs(x = "seconds", title = "Time to check", y = element_blank())

Machines (y axis) vs check time (seconds, x axis), violing plot usually around 100 seconds.

Distribution of checking time on each machine.

This means that to check all the packages on the multiple flavors with a single CPU would take 340 days.

tc |> 
  filter(!is.na(T_total)) |> 
  ggplot() +
  geom_violin(aes(T_total, Flavor)) +
  scale_x_log10() +
  labs(x = "seconds", title = "Time to check and install", y = element_blank())

Machines (y axis) vs total time (seconds, x axis), violing plot usually around 100 seconds.

Distribution of total time on each machine.

This means that to install and check all the packages with a single CPU would take 420 days.

I don’t know the computational cost of 266 days of CPU (every day), but a rough calculation of 2.5 cents per hour means 251.92 dollars daily dedicated to this.

tc |> 
  group_by(Package) |> 
  summarize(Versions = n_distinct(Version)) |> 
  ungroup() |> 
  count(Versions, name = "Packages", sort = TRUE) |> 
  flextable() |> 
  autofit()

Versions	Packages
1	22,926
2	481
3	6

This was surprising, but sometimes checks have multiple versions. Probably when a new version is added and the system don’t catch it for a certain machine.

tc |> 
  group_by(Package) |> 
  summarize(Flavors = n_distinct(Flavor)) |> 
  ungroup() |> 
  count(Flavors, name = "Packages", sort = TRUE) |> 
  flextable() |> 
  autofit()

Flavors	Packages
14	23,222
13	69
11	54
10	35
3	12
12	11
5	6
9	3
7	1

Similarly, often packages are only tested on few configurations.

Combining both we can have packages with few configurations that have multiple versions being tested.

tc |> 
  group_by(Package) |> 
  summarize(Versions = as.character(n_distinct(Version)),
            Flavors = n_distinct(Flavor)) |> 
  ungroup() |> 
  count(Flavors, Versions, name = "Packages") |> 
  ggplot() +
  geom_tile(aes(Flavors, Versions, fill = log10(Packages))) +
  scale_x_continuous(expand = expansion())

Flavors of machines and versions of packages

Most packages are just tested one version.

But focusing on those that have just one version of the package being tested, most of the machines have packages either OK or with some notes.

man_colors <- c("OK" = "green", "NOTE" = "darkgreen", 
                "WARNING" = "yellow", "ERROR" = "red", "FAILURE" = "black")
tc |> 
  group_by(Package) |> 
  filter(n_distinct(Version) == 1) |> 
  ungroup() |> 
  group_by(Flavor) |> 
  count(Status, name = "packages") |> 
  mutate(perc = packages/sum(packages),
         Status = forcats::fct_relevel(Status, names(man_colors))) |> 
  ggplot() + 
  geom_col(aes(perc, Flavor, fill = Status)) +
  scale_x_continuous(expand = expansion(), labels = scales::percent_format()) +
  scale_fill_manual(values = man_colors) +
  labs(title = "Packages check status", x = element_blank())

On the vertical axis the machine, on the horitzonal axis the packages colored by the status.

Most frequent status is OK or NOTE on all machines.

If we look at the most frequent status report for packages we can see this table:

ts <- tc |> 
  group_by(Package) |> 
  filter(n_distinct(Version) == 1) |> 
  count(Status, name = "flavors") |> 
  ungroup() |> 
  tidyr::pivot_wider(values_from = flavors, names_from = Status, 
                     values_fill = 0) |> 
  count(OK, NOTE, WARNING, ERROR, FAILURE, name = "packages", sort = TRUE)
download.file("https://cran.r-project.org/web/packages/packages.rds", 
              destfile = "packages.RDS") # From the help page
ap <- readRDS("packages.RDS") |> 
  as.data.frame() |> 
  distinct(Package, .keep_all = TRUE)
ap_bioc <- available.packages(repos = BiocManager::repositories()[1:5])
ap_bioc <- cbind(ap_bioc, Additional_repositories = NA)
ap_colm <- intersect(colnames(ap), colnames(ap_bioc))
ap <- rbind(ap[, ap_colm], ap_bioc[, ap_colm])
head(ts) |> 
  flextable() |> 
  autofit()

OK	NOTE	packages
14	0	13,433
12	2	4,391
11	3	1,523
0	14	941
9	5	786
10	4	297

We can see that the most common occurrences are some sort of OK and notes on checks. We can also check the official results on CRAN.

We can see that 0.77%, 0.68%, 0.18%, 0.11%, 0.01% of packages pass all checks without notes.

Now let’s see which of the notes or failures are due to intermittent issues.

Compare

First we need to make sure that we compare the right configurations. They must be the same machine, the same R version and the same svn revision between yesterday and today.

# Compare the previous flavor with today's
m_flavor <- which(flavors_df$flavors %in% old_flavors$flavors)
m_version <- which(flavors_df$version %in% old_flavors$version)
m_revision <- which(flavors_df$revision %in% old_flavors$revision)
tm <- table(c(m_flavor, m_version, m_revision))
compare <- flavors_df$flavors[tm == 3] # Only missing the packages version

All changes

Next, compare the status of the packages if the version of the package is the same.

# Find package on the flavors to compare that haven't changed versions
library("dplyr")
tcc <- filter(tc, Flavor %in% compare) |> 
  select(Flavor, Package, Version, Status) |> 
  arrange(Flavor, Package)
ycc <- filter(yc, Flavor %in% compare) |> 
  select(Flavor, Package, Version, Status) |> 
  arrange(Flavor, Package)

all_checks <- merge(tcc, ycc, by = c("Flavor", "Package"), 
                    suffixes = c(".t", ".y"), all = TRUE) 

possible_packages <- all_checks |> 
  filter(Version.t == Version.y & # Same version
           Status.t != Status.y & # Different status
           !is.na(Status.y) & # No new version or removed package
           !is.na(Status.t)) |> 
  rename(Today = Status.t, Yesterday = Status.y)
possible_packages |> 
  select(Package, Flavor, Today, Yesterday, -Version.t, -Version.y) |> 
  arrange(Package, Flavor) |> 
  flextable() |> 
  autofit()

Package	Flavor	Today	Yesterday
AER	r-release-windows-x86_64	NOTE	OK
AutoWMM	r-devel-linux-x86_64-debian-clang	OK	FAILURE
AutoWMM	r-release-linux-x86_64	OK	FAILURE
BayesGP	r-devel-linux-x86_64-debian-clang	OK	NOTE
BayesTwin	r-release-windows-x86_64	WARNING	OK
BayesX	r-release-windows-x86_64	WARNING	OK
BigVAR	r-release-windows-x86_64	ERROR	OK
BioVenn	r-release-windows-x86_64	OK	ERROR
CTTinShiny	r-release-windows-x86_64	WARNING	OK
CohortCharacteristics	r-release-linux-x86_64	ERROR	OK
CohortCharacteristics	r-release-windows-x86_64	ERROR	OK
DIscBIO	r-oldrel-windows-x86_64	ERROR	OK
DIscBIO	r-release-windows-x86_64	OK	ERROR
DeSciDe	r-devel-linux-x86_64-fedora-clang	ERROR	OK
DeSciDe	r-oldrel-windows-x86_64	ERROR	OK
DeSciDe	r-release-linux-x86_64	ERROR	OK
Deducer	r-release-windows-x86_64	ERROR	OK
DiSCos	r-oldrel-windows-x86_64	ERROR	OK
EcoEnsemble	r-release-linux-x86_64	ERROR	OK
FDboost	r-devel-linux-x86_64-fedora-clang	WARNING	OK
FFdownload	r-release-windows-x86_64	ERROR	OK
FRK	r-devel-linux-x86_64-debian-clang	OK	NOTE
GISINTEGRATION	r-release-windows-x86_64	WARNING	OK
GWRLASSO	r-oldrel-windows-x86_64	OK	ERROR
GeoAdjust	r-devel-linux-x86_64-debian-clang	OK	NOTE
JFE	r-release-windows-x86_64	ERROR	OK
KOFM	r-release-windows-x86_64	OK	ERROR
KnowBR	r-release-windows-x86_64	WARNING	NOTE
LocalCop	r-devel-linux-x86_64-debian-clang	OK	NOTE
MLBC	r-devel-linux-x86_64-debian-clang	OK	NOTE
MLZ	r-devel-linux-x86_64-debian-clang	OK	NOTE
MSUthemes	r-release-linux-x86_64	ERROR	OK
MatrixHMM	r-release-windows-x86_64	OK	ERROR
NMsim	r-devel-linux-x86_64-fedora-clang	NOTE	OK
NeEDS4BigData	r-oldrel-windows-x86_64	ERROR	NOTE
Polychrome	r-release-linux-x86_64	NOTE	OK
ProbeDeveloper	r-devel-linux-x86_64-fedora-clang	ERROR	OK
ProbeDeveloper	r-release-linux-x86_64	ERROR	OK
QFASA	r-devel-linux-x86_64-debian-clang	OK	NOTE
RCPA	r-oldrel-windows-x86_64	OK	ERROR
RCPA	r-release-windows-x86_64	ERROR	OK
RCurl	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
RMVL	r-devel-linux-x86_64-debian-clang	OK	ERROR
RSAGA	r-release-windows-x86_64	WARNING	OK
RuHere	r-release-windows-x86_64	ERROR	OK
SAMtool	r-devel-linux-x86_64-debian-clang	OK	NOTE
SDPDmod	r-release-windows-x86_64	ERROR	OK
SPAS	r-devel-linux-x86_64-debian-clang	OK	NOTE
SSAforecast	r-release-windows-x86_64	OK	WARNING
SpatialGEV	r-devel-linux-x86_64-debian-clang	OK	NOTE
SuperLearner	r-oldrel-windows-x86_64	FAILURE	OK
TCGAretriever	r-release-windows-x86_64	ERROR	OK
TaxaNorm	r-oldrel-windows-x86_64	ERROR	OK
TriMatch	r-devel-linux-x86_64-debian-clang	ERROR	OK
TriMatch	r-devel-linux-x86_64-fedora-clang	ERROR	OK
TriMatch	r-release-linux-x86_64	ERROR	OK
WeightedTreemaps	r-release-windows-x86_64	ERROR	OK
agregR	r-release-linux-x86_64	ERROR	OK
annotaR	r-devel-linux-x86_64-debian-clang	ERROR	OK
annotaR	r-release-linux-x86_64	OK	ERROR
asymLD	r-release-linux-x86_64	ERROR	OK
autoGO	r-devel-linux-x86_64-debian-clang	WARNING	OK
backbone	r-release-windows-x86_64	ERROR	OK
bage	r-devel-linux-x86_64-debian-clang	OK	NOTE
bayesMeanScale	r-devel-linux-x86_64-debian-clang	ERROR	OK
bayesMeanScale	r-release-linux-x86_64	ERROR	OK
bayesMeanScale	r-release-windows-x86_64	ERROR	OK
bcRP	r-oldrel-windows-x86_64	ERROR	OK
beezdemand	r-devel-linux-x86_64-debian-clang	OK	NOTE
bigmds	r-oldrel-windows-x86_64	ERROR	OK
biomartr	r-oldrel-windows-x86_64	ERROR	OK
cartographer	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
cffr	r-oldrel-windows-x86_64	OK	ERROR
chopin	r-release-windows-x86_64	ERROR	OK
cli	r-oldrel-windows-x86_64	ERROR	OK
clustTMB	r-devel-linux-x86_64-debian-clang	OK	NOTE
coat	r-devel-linux-x86_64-debian-clang	OK	NOTE
cricketdata	r-release-linux-x86_64	ERROR	OK
crosstable	r-oldrel-windows-x86_64	ERROR	OK
curl	r-oldrel-windows-x86_64	ERROR	NOTE
dartR.sexlinked	r-release-windows-x86_64	WARNING	OK
depmixS4	r-devel-linux-x86_64-debian-clang	OK	NOTE
disaggregation	r-devel-linux-x86_64-debian-clang	OK	NOTE
dlookr	r-release-linux-x86_64	ERROR	OK
dplyr	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
dplyr	r-release-windows-x86_64	ERROR	NOTE
dsdp	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
dsem	r-devel-linux-x86_64-debian-clang	OK	NOTE
duckplyr	r-release-linux-x86_64	ERROR	OK
duckplyr	r-release-windows-x86_64	ERROR	OK
duckspatial	r-release-linux-x86_64	ERROR	OK
duckspatial	r-release-windows-x86_64	ERROR	OK
dynamAedes	r-devel-linux-x86_64-debian-clang	ERROR	NOTE
dynamAedes	r-release-linux-x86_64	ERROR	OK
easyPSID	r-release-windows-x86_64	WARNING	OK
ecb	r-oldrel-windows-x86_64	OK	ERROR
ediblecity	r-oldrel-windows-x86_64	OK	ERROR
ef	r-devel-linux-x86_64-debian-clang	OK	NOTE
enerscape	r-devel-linux-x86_64-debian-clang	OK	WARNING
envi	r-release-windows-x86_64	ERROR	OK
epiDisplay	r-release-windows-x86_64	WARNING	OK
epigrowthfit	r-devel-linux-x86_64-debian-clang	OK	NOTE
etwfe	r-devel-linux-x86_64-debian-clang	ERROR	OK
expectreg	r-release-windows-x86_64	WARNING	OK
filearray	r-devel-linux-x86_64-debian-clang	NOTE	ERROR
fishmethods	r-devel-linux-x86_64-debian-clang	OK	NOTE
flir	r-oldrel-windows-x86_64	OK	ERROR
florabr	r-release-windows-x86_64	ERROR	OK
fossil	r-release-windows-x86_64	WARNING	OK
fractalforest	r-release-windows-x86_64	ERROR	OK
gander	r-release-windows-x86_64	ERROR	OK
gghdx	r-release-linux-x86_64	ERROR	OK
ggtaxplot	r-release-linux-x86_64	ERROR	OK
ggtaxplot	r-release-windows-x86_64	ERROR	OK
gllvm	r-devel-linux-x86_64-debian-clang	OK	NOTE
glmMisrep	r-release-windows-x86_64	ERROR	OK
glmmTMB	r-devel-linux-x86_64-debian-clang	OK	NOTE
gsheet	r-oldrel-windows-x86_64	OK	ERROR
hereR	r-release-linux-x86_64	FAILURE	ERROR
hereR	r-release-windows-x86_64	OK	ERROR
hicp	r-oldrel-windows-x86_64	OK	ERROR
hmmTMB	r-devel-linux-x86_64-debian-clang	OK	NOTE
houba	r-devel-linux-x86_64-debian-clang	OK	ERROR
hsrecombi	r-release-linux-x86_64	OK	ERROR
hsrecombi	r-release-windows-x86_64	ERROR	OK
icosa	r-release-windows-x86_64	ERROR	OK
immunarch	r-release-windows-x86_64	OK	ERROR
jackstraw	r-release-linux-x86_64	ERROR	OK
jsonlite	r-oldrel-windows-x86_64	ERROR	OK
lattice	r-oldrel-windows-x86_64	ERROR	OK
lazyeval	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
leapp	r-release-windows-x86_64	WARNING	NOTE
lme4	r-release-windows-x86_64	ERROR	OK
magrittr	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
mapi	r-release-windows-x86_64	ERROR	OK
mascarade	r-devel-linux-x86_64-fedora-clang	ERROR	OK
mathml	r-devel-linux-x86_64-debian-clang	ERROR	OK
meme	r-release-linux-x86_64	ERROR	OK
mixKernel	r-release-windows-x86_64	OK	ERROR
mkin	r-release-windows-x86_64	OK	ERROR
mmrm	r-devel-linux-x86_64-debian-clang	OK	NOTE
mongolite	r-release-linux-x86_64	OK	ERROR
motif	r-release-windows-x86_64	ERROR	OK
multiSA	r-release-linux-x86_64	WARNING	OK
multilandr	r-release-windows-x86_64	ERROR	OK
newIMVC	r-devel-linux-x86_64-debian-clang	ERROR	OK
nimble	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
nimble	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
nmarank	r-release-windows-x86_64	ERROR	OK
normfluodbf	r-release-windows-x86_64	WARNING	OK
panelsummary	r-devel-linux-x86_64-debian-clang	ERROR	NOTE
parameters	r-oldrel-windows-x86_64	ERROR	NOTE
phylosem	r-devel-linux-x86_64-debian-clang	OK	NOTE
pkr	r-release-windows-x86_64	WARNING	OK
planr	r-devel-linux-x86_64-debian-clang	ERROR	OK
planr	r-release-linux-x86_64	ERROR	OK
precondition	r-devel-linux-x86_64-debian-clang	WARNING	NOTE
qbrms	r-devel-linux-x86_64-debian-clang	OK	NOTE
qol	r-devel-linux-x86_64-debian-clang	ERROR	OK
rATTAINS	r-oldrel-windows-x86_64	OK	ERROR
rATTAINS	r-release-windows-x86_64	ERROR	OK
rAccess	r-release-windows-x86_64	OK	ERROR
ravecore	r-devel-linux-x86_64-debian-clang	OK	ERROR
ravepipeline	r-devel-linux-x86_64-debian-clang	OK	ERROR
rbranding	r-release-linux-x86_64	ERROR	OK
rdomains	r-release-linux-x86_64	OK	ERROR
refseqR	r-oldrel-windows-x86_64	ERROR	OK
renv	r-devel-linux-x86_64-fedora-clang	WARNING	OK
renv	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
restatapi	r-oldrel-windows-x86_64	OK	ERROR
restfulr	r-oldrel-windows-x86_64	OK	ERROR
riskmetric	r-release-windows-x86_64	ERROR	OK
rlang	r-oldrel-windows-x86_64	ERROR	OK
rodeo	r-oldrel-windows-x86_64	OK	ERROR
rolog	r-devel-linux-x86_64-debian-clang	ERROR	OK
roxygen2	r-oldrel-windows-x86_64	ERROR	OK
rpanel	r-oldrel-macos-x86_64	OK	ERROR
rsatscan	r-release-windows-x86_64	WARNING	OK
rsubgroup	r-release-windows-x86_64	WARNING	OK
rswipl	r-devel-linux-x86_64-debian-clang	ERROR	OK
runonce	r-devel-linux-x86_64-debian-clang	ERROR	OK
salmonMSE	r-release-linux-x86_64	WARNING	OK
scPOEM	r-release-linux-x86_64	ERROR	OK
sdmTMB	r-devel-linux-x86_64-debian-clang	OK	NOTE
separationplot	r-release-windows-x86_64	WARNING	OK
shinyscholar	r-release-windows-x86_64	ERROR	OK
showtext	r-release-linux-x86_64	ERROR	OK
snvecR	r-release-linux-x86_64	OK	ERROR
softclassval	r-devel-linux-x86_64-debian-clang	OK	NOTE
sovereign	r-release-windows-x86_64	ERROR	OK
spMaps	r-release-windows-x86_64	ERROR	OK
spatstat.random	r-devel-linux-x86_64-fedora-clang	WARNING	NOTE
srcpkgs	r-release-linux-x86_64	ERROR	OK
srcpkgs	r-release-windows-x86_64	ERROR	OK
ssdtools	r-devel-linux-x86_64-debian-clang	OK	NOTE
stelfi	r-devel-linux-x86_64-debian-clang	OK	NOTE
stochvolTMB	r-devel-linux-x86_64-debian-clang	OK	NOTE
tibble	r-release-windows-x86_64	ERROR	OK
tidyterra	r-release-windows-x86_64	OK	ERROR
tinyVAST	r-devel-linux-x86_64-debian-clang	OK	NOTE
tmbstan	r-devel-linux-x86_64-fedora-gcc	WARNING	NOTE
tsdistributions	r-devel-linux-x86_64-debian-clang	OK	NOTE
tsissm	r-devel-linux-x86_64-debian-clang	OK	NOTE
unmarked	r-devel-linux-x86_64-debian-clang	OK	NOTE
vctrs	r-release-windows-x86_64	NOTE	ERROR
xaringanthemer	r-release-linux-x86_64	ERROR	OK
yaml	r-oldrel-windows-x86_64	ERROR	OK

If the machine and R versions is the same but the check of the package is different there might be some discrepancy between the dependencies.

# Extract dependencies
dependencies <- package_dependencies(unique(possible_packages$Package),
                                     # Should it check all the recursive dependencies or only direct?
                                     db = ap, # Only considering those dependencies on CRAN and Bioconductor but not any Additional_repositories. 
                                     recursive = TRUE, 
                                     which = c("Depends", "Imports", "LinkingTo", "Suggests"))

# Prepare to compare versions (as they are sorted by everything else we can compare directly)
intermittent_failures <- rep(FALSE, length(dependencies))
names(intermittent_failures) <- names(dependencies)
dep_0 <- lengths(dependencies) == 0
intermittent_failures[dep_0] <- TRUE

If they do not have any recursive dependency on Depends, Imports, LinkingTo and Suggests they might be have some intermittent problems on the packages. These is only on dependencies on CRAN and Bioconductor but not in other additional repositories (There are 187 packages with additional repositories).

If they have some dependencies and those dependencies didn’t change as far as we can tell then there might be some problems with random numbers or connectivity.

for (pkg in names(intermittent_failures[!intermittent_failures])) {
  dep <- dependencies[[pkg]]
  fl <- possible_packages$Flavor[possible_packages$Package == pkg]
  intermittent_failures[pkg] <- all_checks |> 
    filter(Package %in% dep,
           Flavor %in% fl,
           Version.t == Version.y,
           Status.t != Status.y) |> 
    nrow() == 0 # If packages outside || any(!dep %in% rownames(ap)) 
}
packages <- names(intermittent_failures)[intermittent_failures]

We finally show the differences on the status of those without any dependency change on version or status¹:

keep_files <- filter(possible_packages, Package %in% packages) |> 
  merge(y = flavors_df, by.x = "Flavor", by.y = "flavors", all.x = TRUE, all.y = FALSE) |> 
  select(Package, Flavor, Version = Version.t, R_version = r_version, OS = os, 
         architecture, other, version, revision) |> 
  mutate(Date = Sys.time())

if (nrow(keep_files >= 1)) {
  write.csv(keep_files, 
            paste0("cran-failing-", format(Sys.time(), "%Y%m%dT%H%M"), ".csv"),
            row.names = FALSE,
            quote = FALSE,
  )
}

filter(possible_packages, Package %in% packages) |> 
  select(Package, Flavor, Today, Yesterday, -Version.t, -Version.y) |> 
  flextable() |> 
  autofit()

Package	Flavor	Today	Yesterday
RMVL	r-devel-linux-x86_64-debian-clang	OK	ERROR
rsubgroup	r-release-windows-x86_64	WARNING	OK

Conclusion

cat("There are no packages detected with differences between yesterday and today attributable to intermittent failures.\n")
knitr::knit_exit()

cat("This suggests that these packages might have some problems with random numbers or connectivity:\n\n")

This suggests that these packages might have some problems with random numbers or connectivity:

if (any(dep_0)) {
  cat("\n## Packages with dependencies\n\n")
  cat(paste0(" - ", sort(intersect(packages, 
                                   names(dependencies)[dep_0])), "\n"), sep = "")
  cat("\n## Packages without dependencies\n\n")
  cat(paste0(" - ", sort(intersect(packages,
                                   names(dependencies)[!dep_0])), "\n"), sep = "")
  
} else {
  cat(paste0(" - ", sort(packages), "\n"), sep = "")
}

Packages with dependencies

RMVL

Packages without dependencies

rsubgroup

I think a new version might not propagate to check other packages until 24 hours later as checks might have already started for that day.↩︎

Packages intermittent failtures

Lluís Revilla Sancho

2026-03-21