Daniel A.的貼文

Methods for Evaluating GIMs The challenge of evaluating these powerful models is to decipher the essence of these creations. Let's dive into the interesting methodologies and metrics that form the core of this captivating evaluation experience: Human Evaluation: The human verdict adds a subjective touch to the evaluation. Despite its reliability, this kind of evaluation has some drawbacks to consider: It is expensive, time-consuming, and prone to bias. Therefore, it is usually considered as a supplementary method for evaluating these models. Pixel-Based Metrics: An alternative method for assessing generative models involves the comparison of generated images with real ones from the same domain, utilizing pixel-based metrics like mean squared error (MSE), peak signal-to-noise ratio or PSNR, or structural similarity index or SSIM. These metrics dig deep into a pixel-level, taking into account that the closer the pixels, the higher the image quality. However, pixel-based metrics also have some limitations, including sensitivity to image transformations, ignoring high-level semantic features, and overlooking the aspects of diversity and innovation. Feature-Based Metrics: If we dive deep beyond pixels, there are neural networks, such as convolutional neural networks CNNs or GANs. And these kind of networks are responsible for finding high-level features, like shapes, textures, colors, styles, etc. The metrics related to this method are: Inception score (IS), Fréchet inception distance (FID), and perceptual path length or PPL. They compare the feature distributions of the generated and real images, and determine how well this model preserves the quality and diversity of the original domain. Task-based Metrics: Evaluating generative models can also involve utilizing task-oriented metrics, and gauging how well the generated images serve downstream functions like classification, segmentation, captioning, or retrieval. These metrics offer insights into the practicality and suitability of the generative model for specific tasks and domains. Examples of task-based metrics include classification accuracy, segmentation accuracy, captioning BLEU score, or retrieval precision and recall. However, it's important to acknowledge that the effectiveness of task-based metrics hinges on the choice and performance of downstream models and may not encompass the broader aspects of image generation. Thanks Jesús López Baeza-Rojano for this awesome post! Post link in the comments

🔔 The human visual system is resilient to geometric alterations. When observing objects visually, these entities frequently experience changes like rotation & translation, leading to variations in the perceived image. Yet, humans possess the innate ability to categorize these objects with precision. A resilient classification system must inherently possess the capability to process information concerning anticipated geometric modifications. This can be achieved either overtly, through a structured design, or covertly, by leveraging invariant attributes. The question then arises: How is this knowledge manifested within the human psyche? 📯 Extensive research by experts in the field has delved into this very inquiry. It has been ascertained that humans possess the ability to extrapolate this knowledge to unfamiliar yet analogous categories. For instance, when confronted with unfamiliar categories or orientations, humans exhibit the capacity to assimilate invariance over a duration. We also demonstrate adaptability to unconventional orientations through cognitive processes like mental visualization. 📢 On the other hand, contemporary classification systems rooted in deep learning exhibit fragility. Despite their unparalleled accuracy on structured datasets, their performance in real-world scenarios remains inconsistent, often leading to detrimental consequences in systems dependent on computer vision. Hence, the pursuit of robust classification remains at the forefront of computer vision research. The central question posed is: Is it feasible to emulate the adaptability, generalizability & dynamic invariance observed in humans within artificial neural frameworks? 🎙 Certain transformations can be intrinsically integrated into the system's design. Such revelations have paved the way for groundbreaking methodologies like CNNs. But this modus operandi imposes stringent design constraints, limiting its universal applicability. An alternative path to resilience is the augmentation of data. By subjecting input data to a series of predetermined transformations, neural networks are trained to execute tasks with reliability. Yet, this method's reliability is questionable, especially when faced with distributional shifts. ❄ Traditional methodologies dictate invariances based on pre-existing transformation sets. But the precise set of invariances is often indeterminable, leading to potential discrepancies. ⚡ Emerging methodologies strive to discern beneficial invariances. Yet, its inability to represent multi-modal or joint distributions limits its scope. There's a new research from UC Berkeley, University of Michigan, Google Research. By conceptualizing the correlation between the perceived image & its classification as a graphical model, they introduce a paradigm shift in robust classification. The empirical studies indicate that attributes such as adaptability and generalizability are inherent within this framework, offering avenues for exploration.

In the age of artificial intelligence, it is the financial institutions that can most effectively harness the power of machine learning that will have a competitive advantage. At J.P Morgan, leading AI specialist Dr. Zhen Zeng, who spoke to our MSBA cohort recently, dedicates her expertise to exploring how existing machine learning models can be adapted to novel use cases in financial forecasting. Dr. Zeng's groundbreaking research is centered around the innovative application of Convolutional Neural Networks (CNNs) and inpainting techniques--traditionally used to complete missing segments of images--to predicting financial time series data. By integrating this novel strategy with sentiment analysis, which involves evaluation of social media discourse and the sentiment expressed in news publications to understand the underlying emotional tone of textual data, Dr. Zeng further hones the precision of her financial predictive models. This cutting-edge synthesis facilitates the integration of investor sentiment and general market mood into the models--factors that are recognized as pivotal indicators of financial market movements and which provide indispensable context to the quantitative data processed by neural networks. Dr. Zeng's approach consequently fosters a more comprehensive grasp of market dynamics, thus paving the way for the development of subtler and more informed investment strategies. We extend our deepest thanks to Dr. Zhen Zeng for generously sharing her exciting research with our Artificial Intelligence class and to Professor Rachel Chung for her role in orchestrating the event.

The closest known and feasible means of replicating the human brain’s connectome and data structure capacities is not the Anthropomorphically named “neural network”. Rather, it is a graph database. For example, a pyramidal neuron in the human brain has an average of around 6,000 dendritic spines (~1,200 to 20,000 range) receiving excitatory impulses. Over time, all of those connections dynamically change, with some being pruned and new connections forming, as others are modulated up or down. No hardware yet created or visible on humanity’s near-term horizon of development can mirror these dynamics to any meaningful degree. So, how can this be accomplished in software? A graph database can be designed to accomplish this, as any node in a database can be connected to any other node, with every connection (“surface”) having contextual information, which can include motivational system data, of-type relationships, and so on. These connections and nodes can all be dynamically and selectively updated. Each node can also contain and/or reference virtually any kind of data. These graph databases can also grow dynamically in scale, something that neither the human brain nor neural networks are capable of. This scalability also allows them to overcome the Cognitive Bias versus Complexity trade-off, something the human cognitive architecture is fundamentally incapable of. Our last research system grew from a graph under 1 gigabyte in size to over 1.6 terabytes. This type of growth requires a human-like motivational system embedded in the graph, as well as a working cognitive architecture designed to process that additional information. Otherwise, navigation, exploration, updating, goals, interests, and halting/switching would all become intractable problems. Human brains must fit within human skulls. Neural networks are trained to operate at fixed scales, and they lack the human motivational system and a framework that could handle it, as well as the ability to “learn” and “reason” in any meaningful sense. Effectively, everyone trying to get “#AGI” out of neural networks is like a toddler trying to shove a square block through a round hole. Some have pointed to more money being spent on cigarette advertisements than humanity spends on mitigating existential risks. An example that is closer to home for the #AI industry could be the $20m+ invested in several new “AGI companies” with nothing but a vague idea and a pedigree, the $44m in X.ai, or the hundreds of millions and even billions of dollars invested in #chatbot companies like OpenAI, Inflection, Mistral, and so on. These companies are cigarette advertisements. The toddler that aims to push a square block through a round hole is playing in the nuclear missile silo. How long does humanity intend to leave them unattended? #ethics #bias

A New Way To Predict Ship-Killing Rogue Waves: AI models can find patterns and make predictions, but their reasoning is often inscrutable. This "black box" issue makes AI less reliable and less scientifically useful. However, a team led by Dion Hafner (a computer scientist at the University of Copenhagen) devised a clever neural network to predict rogue waves. By restricting inputs to meaningful wave measurements and tracing how they flowed through the network, the team extracted a simple five-part equation encapsulating the AI's logic. Economist adds: To generate a human-comprehensible equation, the researchers used a method inspired by natural selection in biology. They told a separate algorithm to come up with a slew of different equations using those five variables, with the aim of matching the neural network's output as closely as possible. The best equations were mixed and combined, and the process was repeated. The result, eventually, was an equation that was simple and almost as accurate as the neural network. Both predicted rogue waves better than existing models. The first part of the equation rediscovered a bit of existing theory: it is an approximation of a well-known equation in wave dynamics. Other parts included some terms that the researchers suspected might be involved in rogue-wave formation but are not in standard models. There were some puzzlers, too: the final bit of the equation includes a term that is inversely proportional to how spread out the energy of the waves is. Current human theories include a second variable that the machine did not replicate. One explanation is that the network was not trained on a wide enough selection of examples. Another is that the machine is right, and the second variable is not actually necessary. Read more of this story at Slashdot.

To cut a long story short, from the conclusion. ...Statistical inference is the distorted, new eye of the capital's Master... Dixit Matteo Pasquinelli. I persist and sign AI can only explore the Zone de la Pensée, devoid of disruptive Ideas. Therefore, AI is, at best, an attractive innovation but incapable of solving today's complex challenges, let alone those of the Cité de Demain. To overcome the mental reduction built by AI, we must go beyond the limitations of deductive and inductive reasoning. Instead, we must improve our understanding and harness our intuition resonance. That’s why if we don't just want to navigate the Zone de L’Impensée blindly, we need to master the intuition's psychic energy of the Intelligible.

This is a really cool research from anthropic (creators of Claude2 AI), where they demonstrate that "Units of Analysis" (which they call Features) are much better indicators of LLM behaviours than "Individual Neurons" What is still amazing is how even today (Oct 2023) there isn't good and solid understanding of how these LLMs work (this research is a good step in that direction) Nice conclusion too: "...For the first time, we feel that the next primary obstacle to interpreting large language models is engineering rather than science...." For more details see: - full research paper: https://lnkd.in/ecvx_7xJ - A1 Features - https://lnkd.in/ekReS_Z5 - All Features - https://lnkd.in/eh9iYnK5 The last last two links above provide really good insights and visualisation on how LLMs and Transformers work

Thank you to Dinis Cruz for sharing. Some quick GPT assisted analysis: GPT Prompt: How can i use this information to create better prompts? Utilising the information from the article, you could create better prompts by designing them to specifically target monosemantic features, making the outcomes more interpretable. Furthermore, understanding how sparse autoencoders work could help in structuring prompts that are conducive to extracting meaningful and clear features from neural networks, thereby potentially leading to more insightful and understandable responses. Lastly, the visualisations and analyses mentioned in the article might inspire ways to visually represent the behaviour of your models, aiding in better prompt design and analysis.