"Video Influencers: Unboxing the Mystique. (arXiv:2012.12311v2 [cs.LG] UPDATED)" — A study and analysis of YouTube influencers and their unstructured video data across text, audio and images using a novel "interpretable deep learning" framework to determine the effectiveness of their constituent elements in explaining video engagement.

Paper: http://arxiv.org/abs/2012.12311

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Gradient heat map in video fram…

"Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models. (arXiv:2301.13826v1 [cs.CV])" — A process which intervenes in the generative process of diffusion models on the fly during inference time to improve the faithfulness of the generated images to guide the model to refine the cross-attention units to attend to all subject tokens in the text prompt and strengthen - or excite - their activations, encouraging the model to generate all subjects described in the text prompt.

Paper: http://arxiv.org/abs/2301.13826
Code: https://github.com/AttendAndExcite/Attend-and-Excite

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Given a pre-trained text-to-ima…

"Grounding Language Models to Images for Multimodal Generation. (arXiv:2301.13823v1 [cs.CL])" — An efficient method to ground pretrained text-only language models to the visual domain, enabling them to process and generate arbitrarily interleaved image-and-text data. This approach apparently works with any off-the-shelf language model.

Paper: http://arxiv.org/abs/2301.13823

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Our method grounds a language m…

"UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers. (arXiv:2301.13741v1 [cs.CV])" — A universal vison-language Transformer compression framework which can handle multiple generative and discriminative vision-language tasks such as Visual Reasoning, Image Caption, Visual Question Answer, Image-Text Retrieval, Text-Image Retrieval, and Image Classification.

Paper: http://arxiv.org/abs/2301.13741

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Comparison between the Mask-bas…

"DisDiff: Unsupervised Disentanglement of Diffusion Probabilistic Models. (arXiv:2301.13721v1 [cs.CV])" — A new task to take advantage of the remarkable modeling ability of diffusion probabilistic models (DPM) using an unsupervised approach.

Paper: http://arxiv.org/abs/2301.13721

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Illustration of disentanglement…

"Learning Data Representations with Joint Diffusion Models. (arXiv:2301.13622v1 [cs.LG])" — A joint diffusion model that simultaneously learns meaningful internal representations fit for both generative and predictive tasks and which has superior performance across various tasks, including generative modeling, semi-supervised classification, and domain adaptation.

Paper: http://arxiv.org/abs/2301.13622

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Data representation zt in a UNe…

"NP-Match: Towards a New Probabilistic Model for Semi-Supervised Learning. (arXiv:2301.13569v1 [cs.CV])" — Adapting neural processes (NPs) for semi-supervised image classification tasks to arrive at a solution with much less computational overhead, which can save time at both the training and the testing phases.

Paper: http://arxiv.org/abs/2301.13569
Code: https://github.com/jianf-wang/np-match

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Overview of NP-Match: it contai…

"Domain-Generalizable Multiple-Domain Clustering. (arXiv:2301.13530v1 [cs.LG])" — Given unlabeled samples from multiple source domains, an attempt to learn a shared classifier that assigns the examples to various clusters by using the classifier for predicting cluster assignments in a previously unseen domain.

Paper: http://arxiv.org/abs/2301.13530

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Problem statement - given unlab…

"Fourier Sensitivity and Regularization of Computer Vision Models. (arXiv:2301.13514v1 [cs.CV])" — A study of the frequency sensitivity characteristics of deep neural networks using a principled approach due to recent work showing that deep neural networks latch on to the Fourier statistics of training data and show increased sensitivity to Fourier-basis directions in the input.

Paper: http://arxiv.org/abs/2301.13514

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Computing Fourier-sensitivity. …

"Conversational Automated Program Repair" — A method to help developers automatically generate patches for bugs using Large Language Models (LLMs) using a conversational approach for patch generation and validation.

Paper: https://arxiv.org/abs/2301.13246

#AI #NewPaper #MachineLearning #SoftwareEngineering

<<Find this useful? Please boost so that others can benefit too 🙂>>
Overview of conversational APR …

"Continuous Spatiotemporal Transformers. (arXiv:2301.13338v1 [cs.LG])" — A new transformer architecture that is designed for the modeling of continuous systems which guarantees a continuous and smooth output via optimization in Sobolev space.

Paper: http://arxiv.org/abs/2301.13338

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Diagram of CST’s workflow. (A) …

"ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Models. (arXiv:2301.12935v2 [cs.LG] UPDATED)" — An error-robust Adams solver (ERA-Solver), which utilizes the implicit Adams numerical method that consists of a predictor and a corrector.

Paper: http://arxiv.org/abs/2301.12935
Note: The PDF for this version of the paper is currently not available

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
We adopt the pretrained diffusi…

"PromptMix: Text-to-image diffusion models enhance the performance of lightweight networks. (arXiv:2301.12914v2 [cs.CV] UPDATED)" — A method for artificially boosting the size of existing datasets, that can be used to improve the performance of lightweight networks.

Paper: http://arxiv.org/abs/2301.12914

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Overview of PromptMix

"MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models. (arXiv:2210.01820v2 [cs.CV] UPDATED)" — A family of neural networks that build on top of mobile convolution (i.e., inverted residual blocks) and attention which not only enhances the network representation capacity, but also produces better downsampled features.

Paper: http://arxiv.org/abs/2210.01820
Code: https://github.com/google-research/deeplab2

#AI #CV #NewPaper #DeepLearning #MachineLearning

<<Find this useful? Please boost so that others can benefit too 🙂>>
Block comparison. (a) The MBCon…

A total of 89 papers in the cs.CV category on arXiv.org today — 66 new, 23 updated.

Same number of overall papers as yesterday, but a lot more new (vs updated) papers today. On to the reading ...

#AI #CV #NewPapers #DeepLearning #MachineLearning

I’ve dabbled with three different #StableDiffusion GUIs over the last few months (in addition to the ones I coded myself, of course ..). They are:
1. Automatic1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui
2. InvokeAI: https://github.com/invoke-ai/InvokeAI
3. Stable Diffusion UI: https://github.com/cmdr2/stable-diffusion-ui

Personally for me, Automatic1111 has always been harder to get working right. Something or other will break in terms of dependencies, or it will take way too much effort to get working right. So that’s kind of out of the running.

The other two install easily on both macOS and Windows and the installers work right and the GUI is nice in both. So I’ve been trying to decide between the two …

Just installed the latest InvokeAI and it has a model manager which actually lets you do useful stuff via the WebUI and the decision seems an easy one … till of course, Stable Diffusion UI catches up 😛

“There are two seasons in Scotland. June and winter.”
“In Scotland, there's no such thing as bad weather, only the wrong clothes.”

Billy Connolly

I beg to differ. We had all four seasons today - rain, sunshine, hail, high winds, more sun, more showers. He’s right about the clothing though.

This scene in the afternoon near our local shore. Grass, seaweed, rocks, the sea, the cliffs and the clouded sky infused with palest gold. Brodie glowing in his fur coat.

#winter #dogsofmastodon #golden

Every afternoon at 4 o’clock they meet to read together.
#books #reading #robots #ai #aiart #midjourney

If you're into #data or #design, and you're around #Melbourne #Naarm in March, then you might like Data Informed Design conference - #DiD23

It's produced by Place Intelligence, who do good things in the urban design space.

https://did.placeintelligence.ai

cc @FionaGrayPhD

moss on a redbud

Try saying it like a Victorian British exclamation of dismay: "Moss on a redbud, man! What is this country coming to??" 🧐

#mosstodon