Generative Artificial Intelligence, developed within the domain of deep learning, is capable of producing new information such as text, images, or audio. The center of modern artificial intelligence is an optimized prediction, an idea that any provided task, from recognizing a face to composing a poet, can be framed as an optimization problem. In practical terms, this means training a model to minimize a loss function (or maximizing rewards). A loss function mathematically represents the gap between the network's output and the desired outcome.
During the training process, an algorithm known as the backpropagation algorithm computes how wrong the model's output is and how much to adjust it. The network then works backward from the output, calculating the contribution of each parameter to the error and then applying it to individual neurons to optimize how wrong it is. An optimizer like gradient descent uses the information of the loss function to nudge all the weights slightly in the direction that most reduce the loss. By iteratively applying these adjustments across many training examples, the network learns patterns, improving its accuracy by minimizing the loss over time. These three components: the loss function, the backpropagation of error, and the rule to update weights (gradient descent) form a self-correcting feedback loop that is cybernetic in principle: the system continuously measures its performance and adjusts itself in response, steering closer to a goal.
Given image recognition as a practical example, a loss function measures the difference between the generated image and the desired criteria, guiding the model to improve its quality of processing a certain image that aligns with the expected results. I was pondered by the process of learning. Could an algorithm function as a critique? A mathematical sense of judgment of “this is not what I want.” Imagine the myriad of criteria humans use to evaluate a piece of art (its composition, texture, originality, and cultural relevance) being compressed into a computable set of metrics. An AI that optimizes loss function may converge towards solutions that satisfy the numeric criteria yet surprise us in form. However, much is lost in translation when human aesthetic experience is reduced to numbers. As an artistic production never falls into a single quantifiable set of goals, it is often about exploration, context, and even deliberate ambiguity and provocations. Can such qualities also be captured by a loss function? Beatrice Fazi argues that not everything meaningful in art or experience can be predetermined or discretely measured. It reminds me that human intention and algorithmic interpretations are never fully closed; a loss function, therefore, is not a fixed target but a moving interplay between our guidance and the machine's iterative trial-and-error.Sections
- What you see is not inside the machine - The illusion of AI-generated content
- The purple coincidence - Investigating unexpected outcomes in AI feedback loops
- Eliza Effect - Understanding human emotional responses to AI
- Specificity in Imaging - Tracing the lineage of image-making techniques
- Statistical Index and Embeddings - Examining the role of statistical measures in AI