Shakedown Social

0 posts0 participants0 posts today

DurstewitzLabCan time series (TS) <a href="https://mathstodon.xyz/tags/FoundationModels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FoundationModels</a> (FM) like Chronos zero-shot generalize to unseen <a href="https://mathstodon.xyz/tags/DynamicalSystems" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DynamicalSystems</a> (DS)? <a href="https://mathstodon.xyz/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a>No, they cannot!But *DynaMix* can, the first TS/DS foundation model based on principles of DS reconstruction, capturing the long-term evolution of out-of-domain DS: <a href="https://arxiv.org/pdf/2505.13192v1" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://arxiv.org/pdf/2505.13192v1</a>Unlike TS foundation models, DynaMix exhibits <a href="https://mathstodon.xyz/tags/ZeroShotLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ZeroShotLearning</a> of long-term stats of unseen DS, incl. attractor geometry & power spectrum, w/o *any* re-training, just from a context signal. It does so with only 0.1% of the parameters of Chronos & 10x faster inference times than the closest competitor.It often even outperforms TS FMs on forecasting diverse empirical time series, like weather, traffic, or medical data, typically used to train TS FMs. This is surprising, cos DynaMix’ training corpus consists *solely* of simulated limit cycles & chaotic systems, no empirical data at all!And no, it’s neither based on Transformers nor Mamba – it’s a new type of mixture-of-experts architecture based on the recently introduced AL-RNN (<a href="https://proceedings.neurips.cc/paper_files/paper/2024/file/40cf27290cc2bd98a428b567ba25075c-Paper-Conference.pdf" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://proceedings.neurips.cc/paper_files/paper/2024/file/40cf27290cc2bd98a428b567ba25075c-Paper-Conference.pdf</a>), specifically trained for DS reconstruction.Remarkably, DynaMix not only generalizes zero-shot to novel DS, but it can even generalize to new initial conditions and regions of state space not covered by the in-context information.We dive a bit into the reasons why current time series FMs not trained for DS reconstruction fail, and conclude that a DS perspective on time series forecasting & models may help to advance the <a href="https://mathstodon.xyz/tags/TimeSeriesAnalysis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TimeSeriesAnalysis</a> field.

Toni Aittoniemi<a href="https://techhub.social/@Techmeme" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@Techmeme</a> This is the danger of closed sourceThese are knowledge models, and they only output what they are fed withAnd no, they won’t magically develop ’reasoning skills’ and be able to sift through propaganda. NOT when it’s part of the trainingTo think otherwise means you don’t know shit how they workThey obey statistics. Training data for <a href="https://mastodon.green/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://mastodon.green/tags/foundationmodels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#foundationmodels</a> should be subject to public <a href="https://mastodon.green/tags/academic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#academic</a> scrutiny. Otherwise the models are bound to fall for flooding attacks

lkngrrr 🏳️‍🌈🏳️‍⚧️🏴‍☠️<a href="https://hachyderm.io/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a>, <a href="https://hachyderm.io/tags/FoundationModels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FoundationModels</a>, <a href="https://hachyderm.io/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a>, and all of that hokey nonsense shall not appear in my <a href="https://hachyderm.io/tags/robotics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#robotics</a> roadmaps as anything other than a neat research item until it can demonstrate a feasible path to <a href="https://hachyderm.io/tags/FunctionalSafety" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FunctionalSafety</a> or mathematical completeness. I lead <a href="https://hachyderm.io/tags/Product" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Product</a> on the largest mobile-<a href="https://hachyderm.io/tags/robotic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#robotic</a> fleet known to humankind. I will not entrust decisions that could maim or kill to a pile of nondeterminate math prone to “hallucinations” or confabulation.<a href="https://hachyderm.io/tags/ProductManagement" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ProductManagement</a>

Erik JonkerProf. Yoshua Bengio warns in an op-ed for German newspaper Tagesspiegel: exempting foundation models from the AI Act would be both dangerous & economically costly. It would make the AI Act "outdated from day one". <a href="https://background.tagesspiegel.de/digitalisierung/die-eu-droht-eine-einzigartige-chance-zu-verspielen" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://background.tagesspiegel.de/digitalisierung/die-eu-droht-eine-einzigartige-chance-zu-verspielen</a> <a href="https://mastodon.social/tags/EU" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#EU</a> <a href="https://mastodon.social/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://mastodon.social/tags/AIact" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AIact</a> <a href="https://mastodon.social/tags/generativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#generativeAI</a> <a href="https://mastodon.social/tags/foundationmodels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#foundationmodels</a> <a href="https://mastodon.social/tags/Bengio" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bengio</a>

IT NewsUniversity of Chicago researchers seek to “poison” AI art generators with Nightshade - Enlarge (credit: Getty Images) On Friday, a team of researcher... - <a href="https://arstechnica.com/?p=1978501" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://arstechnica.com/?p=1978501</a> <a href="https://schleuss.online/tags/largelanguagemodels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#largelanguagemodels</a> <a href="https://schleuss.online/tags/universityofchicago" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#universityofchicago</a> <a href="https://schleuss.online/tags/adversarialattacks" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#adversarialattacks</a> <a href="https://schleuss.online/tags/foundationmodels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#foundationmodels</a> <a href="https://schleuss.online/tags/machinelearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#machinelearning</a> <a href="https://schleuss.online/tags/aitrainingdata" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#aitrainingdata</a> <a href="https://schleuss.online/tags/imagesynthesis" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#imagesynthesis</a> <a href="https://schleuss.online/tags/datapoisoning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#datapoisoning</a> <a href="https://schleuss.online/tags/nightshade" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#nightshade</a> <a href="https://schleuss.online/tags/aiethics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#aiethics</a> <a href="https://schleuss.online/tags/benzhao" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#benzhao</a> <a href="https://schleuss.online/tags/biz" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#biz</a>⁢ <a href="https://schleuss.online/tags/google" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#google</a> <a href="https://schleuss.online/tags/metaai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#metaai</a> <a href="https://schleuss.online/tags/openai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#openai</a> <a href="https://schleuss.online/tags/aiart" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#aiart</a> <a href="https://schleuss.online/tags/glaze" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#glaze</a> <a href="https://schleuss.online/tags/meta" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#meta</a> <a href="https://schleuss.online/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a>

Harald SackHang in there, my fellow <a href="https://sigmoid.social/tags/knowledgegraph" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#knowledgegraph</a> researchers and practitioners, soon (in a few years) we will reach the "Slope of Enlightenment" ;-) The new Gartner hype cycle for AI positions knowledge graphs right in the middle of the "Through of Disillusionment" ... while placing <a href="https://sigmoid.social/tags/GenerativeAI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GenerativeAI</a> and <a href="https://sigmoid.social/tags/FoundationModels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FoundationModels</a> <a href="https://sigmoid.social/tags/LLMs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LLMs</a> at the peak of the hype <a href="https://www.gartner.com/en/articles/what-s-new-in-artificial-intelligence-from-the-2023-gartner-hype-cycle#:~:text=The%202023%20Gartner%20Hype%20Cycle%E2%84%A2%20for%20Artificial%20Intelligence%20" rel="nofollow noopener noreferrer" target="_blank">https://www.gartner.com/en/articles/what-s-new-in-artificial-intelligence-from-the-2023-gartner-hype-cycle#:~:text=The%202023%20Gartner%20Hype%20Cycle%E2%84%A2%20for%20Artificial%20Intelligence%20</a>(AI,most%20credible%20cases%20for%20investment. <a href="https://sigmoid.social/tags/semanticweb" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#semanticweb</a> <a href="https://sigmoid.social/tags/ai" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ai</a> <a href="https://sigmoid.social/tags/hypecycle" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#hypecycle</a> <a href="https://sigmoid.social/tags/artificialintelligence" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#artificialintelligence</a>

Recent searches

Search options

Administered by:

Server stats:

#foundationmodels