Spyke

Posts

machinelearning·Machine LearningbyDeliverator

BayesFlow: Amortized Bayesian Workflows With Neural Networks

Modern Bayesian inference involves a mixture of computational techniques for estimating, validating, and drawing conclusions from probabilistic models as part of principled workflows for data analysis. Typical problems in Bayesian workflows are the approximation of intractable posterior distributions for diverse model types and the comparison of competing models of the same process in terms of their complexity and predictive performance. This manuscript introduces the Python library BayesFlow for simulation-based training of established neural network architectures for amortized data compression and inference. Amortized Bayesian inference, as implemented in BayesFlow, enables users to train custom neural networks on model simulations and re-use these networks for any subsequent application of the models. Since the trained networks can perform inference almost instantaneously, the upfront neural network training is quickly amortized.

https://arxiv.org/abs/2306.16015Open linkView original on kbin.social

Scientists have finally 'heard' the chorus of gravitational waves that ripple through the universe

NEW YORK (AP) — Scientists have observed for the first time the faint ripples caused by the motion of black holes that are gently stretching and squeezing everything in the universe.

They reported Wednesday that they were able to “hear” what are called low-frequency gravitational waves — changes in the fabric of the universe that are created by huge objects moving around and colliding in space.

“It’s really the first time that we have evidence of just this large-scale motion of everything in the universe,” said Maura McLaughlin, co-director of NANOGrav, the research collaboration that published the results in The Astrophysical Journal Letters.

https://apnews.com/article/gravitational-waves-black-holes-universe-cc0d633ec51a5dc3acb0492baf7f818aOpen linkView original on kbin.social
programming·ProgrammingbyDeliverator

The Curse of Recursion: Training on Generated Data Makes Models Forget

Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public. It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images. In this paper we consider what the future might hold. What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. We refer to this effect as Model Collapse and show that it can occur in Variational Autoencoders, Gaussian Mixture Models and LLMs. We build theoretical intuition behind the phenomenon and portray its ubiquity amongst all learned generative models. We demonstrate that it has to be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in data crawled from the Internet.

https://arxiv.org/abs/2305.17493v2Open linkView original on kbin.social
machinelearning·Machine LearningbyDeliverator

The Curse of Recursion: Training on Generated Data Makes Models Forget

Stable Diffusion revolutionised image creation from descriptive text. GPT-2, GPT-3(.5) and GPT-4 demonstrated astonishing performance across a variety of language tasks. ChatGPT introduced such language models to the general public. It is now clear that large language models (LLMs) are here to stay, and will bring about drastic change in the whole ecosystem of online text and images. In this paper we consider what the future might hold. What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. We refer to this effect as Model Collapse and show that it can occur in Variational Autoencoders, Gaussian Mixture Models and LLMs. We build theoretical intuition behind the phenomenon and portray its ubiquity amongst all learned generative models. We demonstrate that it has to be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in data crawled from the Internet.

https://arxiv.org/abs/2305.17493v2Open linkView original on kbin.social
machinelearning·Machine LearningbyDeliverator

Machine Learning Beginner Info/Resources

MOOCs

Nowadays, there are a couple of really excellent online lectures to get you started. The list is too long to include them all. Every one of the major MOOC sites offers not only one but several good Machine Learning classes, so please check coursera, edX, Udacity yourself to see which ones are interesting to you.

However, there are a few that stand out, either because they're very popular or are done by people who are famous for their work in ML. Roughly in order from easiest to hardest, those are:

Books

The most often recommended textbooks on general Machine Learning are (in no particular order):

Note that these books delve deep into math, and might be a bit heavy for complete beginners. If you don't care so much about derivations or how exactly the methods work but would rather just apply them, then the following are good practical intros:

There are of course a whole plethora on books that only cover specific subjects, as well as many books about surrounding fields in Math. A very good list has been collected by /u/ilsunil here

Deep Learning Resources

Math Resources

Programming Languages and Software

In general, the most used languages in ML are probably Python, R and Matlab (with the latter losing more and more ground to the former two). Which one suits you better depends wholy on your personal taste. For R, a lot of functionality is either already in the standard library or can be found through various packages in CRAN. For Python, NumPy/SciPy are a must. From there, Scikit-Learn covers a broad range of ML methods.

If you just want to play around a bit and don't do much programming yourself then things like Visions of Chaos, WEKA, KNIME or RapidMiner might be of your liking. Word of caution: a lot of people in this subreddit are very critical of WEKA, so even though it's listed here, it is probably not a good tool to do anything more than just playing around a bit. A more detailed discussion can be found here

Deep Learning Software, GPU's and Examples

There are a number of modern deep learning toolkits you can utilize to implement your models. Below, you will find some of the more popular toolkits. This is by no means an exhaustive list. Generally speaking, you should utilize whatever GPU has the most memory, highest clock speed, and most CUDA cores available to you. This was the NVIDIA Titan X from the previous generation. These frameworks are all very close in computation speed, so you should choose the one you prefer in terms of syntax.

Theano is a python based deep learning toolkit developed by the Montreal Institute of Learning Algorithms, a cutting edge deep learning academic research center and home of many users of this forum. This has a large number of tutorials ranging from beginner to cutting edge research.

Torch is a Luajit based scientific computing framework developed by Facebook Artificial Intelligence Research (FAIR) and is also in use at Twitter Cortex. There is the torch blog which contains examples of the torch framework in action.

TensorFlow is a python deep learning framework developed by Google Brain and in use at Google Brain and Deepmind. The newest framework around. Some TensorFlow examples may be found here Do not ask questions on the Google Groups, ask them on stackoverflow

Neon is a python based deep learning framework built around a custom and highly performant CUDA compiler Maxas by NervanaSys.

Caffe is an easy to use, beginner friendly deep learning framework. It provides many pretrained models and is built around a protobuf format of implementing neural networks.

Keras can be used to wrap Theano or TensorFlow for ease of use.

Datasets and Challenges for Beginners

There are a lot of good datasets here to try out your new Machine Learning skills.

Research Oriented Datasets

In many papers, you will find a few datasets are the most common. Below, you can find the links to some of them.

Communities

ML Research

Machine Learning is a very active field of research. The two most prominent conferences are without a doubt NIPS and ICML. Both sites contain the pdf-version of the papers accepted there, they're a great way to catch up on the most up-to-date research in the field. Other very good conferences include UAI (general AI), COLT (covers theoretical aspects) and AISTATS.

Good journals for ML papers are the Journal of Machine Learning Research, the Journal of Machine Learning and arxiv.

Other sites and Tutorials

FAQ

  • How much Math/Stats should I know?

That depends on how deep you want to go. For a first exposure (e.g. Ng's Coursera class) you won't need much math, but in order to understand how the methods really work,having at least an undergrad level of Statistics, Linear Algebra and Optimization won't hurt.

View original on kbin.social

Nestor Makhno — Anarchy's Cossack: The Struggle for Free Soviets in Ukraine 1917-1921 (Free Audiobook)

The phenomenal life of Ukrainian peasant Nestor Makhno (1888–1934) provides the framework for this breakneck account of the downfall of the tsarist empire and the civil war that convulsed and bloodied Russia between 1917 and 1921. As in many of history's chivalric tales, clashes were fought through lightning cavalry charges and bitter hand-to-hand, saber-wielding combat. The combatants were drawn from several camps: Budyenny's Red cavalry, the Don Cossacks and Kuban Cossacks (allied with the Whites), Ukrainian nationalists, and Makhnovist partisans. Makhno, a formidable and daring strategist, headed an army of anarchist insurgents—a popular peasant movement which bore his name.

https://archive.org/details/anarchys-cossackOpen linkView original on kbin.social
vegetarian·VegetarianbyDeliverator

Basic tips, tutorials and recipes (work in progress)

Recipes

https://github.com/hrs/recipes

  • excellent repository of vegetarian/vegan recipes compiled by github.

https://vegetatio.com/content/joyous-living-full-vegetarian-cheese-list

  • list of vegetarian friendly cheese brands.

Common non-vegetarian ingredients

  • Bonito Flakes (dried fish flakes). Traditional flavoring used in Thai, Chinese, Japanese etc., soups & dishes.

  • Bone Marrow. Extracted from animal bones. Used in broths.

  • Carmine (cochineal insect extract). Used as a red food dye (natural red 4, C.I. 75470, or E120).

  • Castoreum (beaver anal glands). Uncommon but is used in vanilla and musk scents/flavorings.

  • Isinglass (fish bladders). Used in beer and wine making. See Barnivore.com for an updated list of vegetarian brands of beers and wines.

  • Gelatin (extracted from animal skin, bones and tissue).

  • Lard (rendered animal fat).

  • Rennet--Used in cheese making. See vegetatio.com for an updated list of vegetarian brands of cheeses.

  • Lipase (pre-gastric tongue root glands of lambs/calves used in cheese making). See vegetatio.com for an updated list of vegetarian brands of cheeses.

  • L-cysteine (feathers/hair). Used in bread making.

  • Lanolin (sheep oil). Vitamin D supplements.

  • Pepsin (pig stomach). See Rennet.

Hidden animal products in popular foods

  • Gelatin Gelatin is produced from pig, cow and chicken bones. (Sometimes fish bones for rare Passover marshmallows.) Anything with marshmallow has gelatin in it. This includes those hard little cereal marshmallows. Gelatin is also found in most gummy candy, some jellybeans, even most Candy Corn. It's also found in everything by the Necco brand of candy (Necco wafers). It's often found in pudding or cheesecake desserts, and it's in Costco chocolate cake. Those shiny tarts with the fruit on top? Gelatin is usually in the clear film on top. Some random foods with gelatin added - instant rice, peanuts, some wine and beer is cleared with it.

  • MacDonald's French Fries BEEF! These contain an ingredient you would not expect in a french fry - cow! Also look out for lard in any refried beans or in Mexican food in general. This may be limited to the USA.

  • Prepared pie crusts More often than not these contain LARD instead of vegetable shortening. At most health food stores you can find veg-friendly alternative, but most at the mega-marts are lard laden.

  • Texas Roadhouse sweet potatoes I thought this was safe - but they're rubbed down in bacon grease before cooking.

  • Cheese rennet Unless it says vegetable rennet you can assume it's animal rennet, made from the stomach of cows, making your cheese. I don't know how much rennet stays in the final product (or if it is flushed with the whey) but it's good to know.

  • Hidden anchovies Lots of items like sauces, dressings, and flavorings, contain Worcestershire sauce, which itself contains anchovies.

  • Chicken stock/fat I find this in so many products, from soups, chips, rice, noodle dishes, etc. It's one you can find almost anywhere.

Vegetarian Protein Guide

  • Quinoa Protein: 8 grams per 1-cup serving, cooked

  • Buckwheat Protein: 6 grams per 1-cup serving, cooked

  • Soy Protein: 12 grams per ½-cup serving (firm tofu), 15 grams per 3-ounce serving (tempeh), 18 grams per 3.5-ounce serving (natto), 17 grams per 1-cup serving (edamame)

  • Mycoprotein (Quorn) Protein: 13 grams per ½-cup serving

  • Rice and Beans Protein: 8 grams per 1-cup serving

  • Hummus and pita Protein: 7 grams per 1 whole-wheat pita and 2 tablespoons hummus

  • Spirulina Protein: 4 grams per 1 tablespoon

  • Peanut butter sandwich Protein: 15 grams per 2-slice sandwich with 2 tablespoons peanut butter

(source https://greatist.com/health/complete-vegetarian-proteins)

View original on kbin.social

You reached the end