Recent searches

No recent searches

Search options

Only available when logged in.
shakedown.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A community for live music fans with roots in the jam scene. Shakedown Social is run by a team of volunteers (led by @clifff and @sethadam1) and funded by donations.

Administered by:

Server stats:

264
active users

shakedown.social: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.2

#reproduciblecomputing

0 posts0 participants0 posts today
Christian Meesters<p><span class="h-card" translate="no"><a href="https://mstdn.science/@dryak" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>dryak</span></a></span> returning from the <a href="https://fediscience.org/tags/isc25" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>isc25</span></a> I will continue this thread with something applicable everywhere, not just on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>HPC</span></a> clusters:</p><p>Workflow runs can crash. There are a number of possible reasons. Snakemake offers a `--rerun-incomple` flag (or short `--ri`) which lets a user resume a workflow.</p><p>This contribution from Filipe G. Viera describes a small fix to stabilize the feature. Not only will incomplete files be removed after a crash, now it is ensured that all metadata with them are deleted too, before resuming: <a href="https://zenodo.org/records/15490098" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15490098</span><span class="invisible"></span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Today tooting from the <a href="https://fediscience.org/tags/ISC25" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ISC25</span></a> - the International Supercomputing Conference. What better opportunity to brag about something I've done to facilitate using GPUs with Snakemake?</p><p>Here is my contribution, simpler job configuration for GPU jobs:</p><p><a href="https://doi.org/10.5281/zenodo.15551797" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">doi.org/10.5281/zenodo.1555179</span><span class="invisible">7</span></a></p><p>Not alone though: Without valuable input of <span class="h-card" translate="no"><a href="https://mstdn.science/@dryak" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>dryak</span></a></span> . Without him, I would have overlooked something crucial.</p><p>And when we talk about reproducible AI, my take is that we ought to consider workflow managers, too. Something which protocols what you have done with little effort.</p><p><a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>This morning, I am travelling to the <a href="https://fediscience.org/tags/isc25" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>isc25</span></a> and hit a minor bug on <a href="https://fediscience.org/tags/researchequals" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>researchequals</span></a>. Hence, no updates in the collection.</p><p>But still a few to describe without adding the latest contributions:</p><p>For instance, this one (<a href="https://zenodo.org/records/15490064" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15490064</span><span class="invisible"></span></a>) by Filipe G. Vieira: a helper function to extract checksums from files to compare with checksums Snakemake was already able to calculate. Really handy!</p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Before I continue uploading - and I do have a couple of more contributions to add to the <a href="https://fediscience.org/tags/ResearchEquals" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ResearchEquals</span></a> collection - first another contribution by Johanna Elena Schmitz and Jens Zentgraf made at the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> </p><p>One difficulty when dealing with a different scientific question: Do I need to re-invent the wheel (read: write a Workflow from scratch?) just to address my slightly different question?</p><p>Snakemake already allowed to incorporate "alien" workflows, even <a href="https://fediscience.org/tags/Nextflow" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Nextflow</span></a> workflows, into desired workflows. The new contribution allows for a more dynamic contribution - with very little changes.</p><p>Check it out: <a href="https://zenodo.org/records/15489694" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15489694</span><span class="invisible"></span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Let's take a look at another contribution of Johanna Elena Schmitz and Jens Zentgraf from the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> </p><p>Snakemake users probably know that </p><p>`$ snakemake [args] --report`</p><p>will generate a self-contained HTML report. Including all plots and <a href="https://fediscience.org/tags/metadata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>metadata</span></a> a researcher's heart longs for.</p><p>Now, why trigger this manually? If the workflow runs successfully, now we can write (or configure):</p><p>`$ snakemake [args] --report-after-run`</p><p>and Snakemake will autogenerate the same report.</p><p>For details see <a href="https://doi.org/10.5281/zenodo.15489764" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">doi.org/10.5281/zenodo.1548976</span><span class="invisible">4</span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <br><a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>One important feature implemented in the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> : Snakemake will calculate file checksums to detect changes. If a file changes, the rule producing it needs to be re-executed when a workflow it re-triggered. But what if a file is too big for reasonable checksum calculation? You do not what to wait forever, after all.</p><p>This contribution describes the implementation of a threshold users may set: <a href="https://doi.org/10.5281/zenodo.15489401" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">doi.org/10.5281/zenodo.1548940</span><span class="invisible">1</span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>One important bug fix during the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> : the config replacement. Now, users can overwrite existing configurations entirely with `--replace-workflow-config`.</p><p>Details: <a href="https://zenodo.org/records/15479268" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="">zenodo.org/records/15479268</span><span class="invisible"></span></a></p><p>More at <a href="https://www.researchequals.com/collections/hm1w-cg/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">researchequals.com/collections</span><span class="invisible">/hm1w-cg/</span></a></p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/openscience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>openscience</span></a></p>
Christian Meesters<p>Did you know? During the <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SnakemakeHackathon2025</span></a> we had a staggering 194 work items!</p><p>It took a while, but now we are gathering contribution reports and present them online as a ResearchEquals (<a href="https://fediscience.org/@ResearchEquals@akademienl.social" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">fediscience.org/@ResearchEqual</span><span class="invisible">s@akademienl.social</span></a>) collection:</p><p><a href="https://www.researchequals.com/collections/hm1w-cg" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">researchequals.com/collections</span><span class="invisible">/hm1w-cg</span></a></p><p>The first 10 are online and I will post some highlights in the coming weeks.</p><p><a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ReproducibleResearch</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>OpenScience</span></a></p>
TrendingLive feeds
About