I’m ashamed to say that I was largely unaware of semiotic
language formation for the bulk of my life – I didn't even hear the term
semiotics until I read Gibson’s Patten
Recognition at the age of 21, hung over, sleep deprived, and jet lagged, a
fine way to read that particular book.
Even after I’d been introduced to the term it remained buried somewhere
in my subconscious for the most part, never even approaching the forefront of
my thought process until I began teaching composition. In composition, which focuses predominantly
on conventional “verbally literate” texts, that is to say texts made up
primarily of words, concerns about visual literacy (which normally deals with texts
composed primarily of pictures) are often given short shrift, if they’re
engaged with at all. When you think about it, that's pretty insane: most people
are far more adept at symbolic or semiotic visual literacy than they are at
verbal-visual literacy. After all,
semiotic literacy engages directly with visual memory through pattern
recognition, without using any of that messy locative memory shortcutting that
our jury-rigged ape brains use to generate visual-verbal literacy
equivalencies. In a sense, that makes it
much more challenging to actually engage with semiotic and symbolic language:
we’re used to just accepting the language of semiotics in the world around us
unconsciously, absorbing information without realizing it. Reading is a conscious act, a challenge for
many people, and fixating on aspects of a process you’re already aware of, a
process you’re forced to consciously work on improving during some phase of your
educational development regardless of who you are, can be much easier than
forcing yourself to analyze the nuances of a process you engage in almost as
reflexively as breathing. But when you discuss
semiotic or visual literacy in concrete terms, students who had trouble with
general literacy begin to acquire deeper critical thinking skills they can
utilize to, in turn, improve weaker skills that they lacked the capacity to
self-assess their progress in previously.
It’s far from a perfect system, and it’s challenging: the study of semiotics
is never really engaged in in a thorough critical framework, and those who do
study it often do so for the sake of manipulating its frames through the
generation of branded material in fields like design and advertising, rather
than the generation of discourse surrounding the response of individuals to
that brand. It's an unfortunate state of
affairs, because breaking down the function of semiotic response mechanisms can
be extremely edifying. By making the
invisible visible to an audience, you’re doing more than just deconstructing
symbols: you’re deconstructing the response-feedback loop surrounding those
symbols.
So what the fuck does any of this have to do with video
games?
More than you’d think.
Video games are essentially engines for instruction. They all but categorically teach players to
apply patterned solutions to problems, often stacking mechanics on top of one
another through sustained play that prompt increasingly complex solutions in
response, generating a feedback patterns promoting general critical thinking
and problem solving skills. Think of Half-Life 2’s helicopter battles. First you’re on a boat. You’ve got no rockets. You’ve got no choice, really, but to run the
fuck away from that god damn helicopter. Alyx is there giving you tips. She’s acting as the voice of the developers
as they teach you how to avoid being straight up murdered by that god damn
flying death machine. After that, you’ll be given a handful of rockets and
asked to fight another helicopter. Here
you learn how to aim and fire while evading, utilizing cover to keep yourself
safe and firing at the helicopter when opportune moments presents themselves.
There’s a steady mix of aiming and dodging and timing there, but the basics are
coming together for you by now: you’re figuring out how to kill these
helicopters, and how badly you can fuck up without thoroughly screwing
yourself. Eventually, you’re forced to
fight these helicopters in different scenarios: in some, you’ll be in mostly
open ground, and your timing as you move from piece of cover to piece of cover
is key. Sometimes, you won’t have access
to any rockets or anti-aircraft weapons at all, and you’ll be forced to use the
grav-gun to fire mines back at the helicopter as it drops them. Sometimes you’ll fight weird
fishy-helicopters that don’t drop mines, and you'll be forced to use the
weapons at your disposal and objects scavenged from your environment. Eventually, you’ll fight them with nothing
but your souped-up grav gun, at which point you’ll only have one weapon to use
against them: energy orbs, which you’ll have been trained in the application of
during a series of exhausting earlier battles.
Half-Life 2
repeats this pattern a number of times across a variety of mechanics, but the
core pattern of challenge-solution-expansion is uniform across these examples. Most games use this pattern to teach players
how to interact with their mechanics, though not all of them are as deliberate
or thorough with their application of this strategy as Half-Life 2. Nor are the
anywhere near as transparent, thanks to Half-Life
2’s seminal and still-enriching “commentary tracks” scattered throughout
the game. Valve took the time not just
to craft these arenas, but to tell us that they were internally called
“arenas,” and that they were oriented around giving players a particular object
lesson, especially in the first half of the game. They made what McGonigal and Gee discussed in
broader terms very specific: they demonstrated the instructive capacity of
games in a directive, measurable way.
They've even got outcomes metrics, something that Gee lacks entirely and
that McGonigal, who I love dearly, is somewhat fuzzy on when she brings them to
bear in her talks.
Video games are also highly reliant on visual languageLet’s
look at Half-Life 2 again, with its
broad scope of art design. Terrain,
construction, and environmental symbolism in Half-Life 2 presents players with a bevy of useful information.
Video games rely predominantly on visual literacy skills, often to the point
that writing becomes a secondary or tertiary concern. Much
of the information we’re given about our world emerges from snippets of visual
language: a vortegaunt standing by a trash can fire, an aging priest wearing a
pair of Chuck Jones originals as he blasts away headcrab zombies with a
shotgun, a horde of zombines pouring out of a troop transport, surging towards
you with grenades in hand. These bits of
visual data all operate on a semiotic layer, either drawing on existing
symbolic or imagistic information from the real world, instructing players in
the meaning of visual frameworks within the game, or building on that understanding
of said visual frameworks to convey important information about the world
players are moving through. Even the
laser sight trail of a sniper rifle is a semiotic clue, providing players with
information about where, when, and how they can move through a space. When you’re given a chance, in Episode 1, to use those sniper rifles to
your advantage through Alyx, the semiotic message is clear: the combine are on
the ropes, and their tools have become yours.
The symbols in the world that represented dangerous obstacles now
represent handy resources that can get you out of a potential jam. The symbol, and its inversion, achieve a
narrative process more effectively than a cutscene or some dialogue might.
Half-Life 2
employs semiotic information to give players instruction and to tell stories in
the world around them, sometimes using subtextual information contained in a
character’s style of dress or the condition of their equipment to give us
background or history about a given individual.
Portal takes it even further,
creating an environment rich with semiotic language, the corruption of which,
as the plot continues, informs players about both the state of their world and
the state of GlaDOS’ authority. As
players begin to strain against it, the veneer begins to peel away, revealing
information about previous occupants of GlaDOS’ experiments, as well as
information about the world you’re in (through the delightful frame of an
abandoned power point presentation, one of the more concrete indicators of
slapdash half considered perspective in the modern world.
All of these references, created by artists and writers, inform
these worlds. They’re a huge contributor
to the kind of world building games are invested in, and a big part of the
particular literacy that games encourage in their participants. The same way
reading a particular kind of book might encourage students to look for certain
patterns of writing to help them locate particular kinds of information,
playing a particular kind of game indoctrinates players towards engaging in a
particular kind of semiotic discussion with their surroundings, as well as the
mechanics of the game itself. A player
who frequently plays RTSes will look at the world in a different way than a
player who plays FPSes, who will look at the world in a different way than
someone who plays RPGs. But all of these
learning structures rely on notions of subtext and symbolism that we don’t
usually discuss, even when we acknowledge their presence, and the generative
proto-linguistic framework that is semiotics is core to effectively discussing
these subjects.
Developing an effective language to that end could be a
major step towards improving games as education tools, literary and artistic
structures, and generating legitimate video game criticism. I think Valve could do some remarkable things
with philosophical and scientific principles given a little push in the right
direction, but right now it doesn’t seem terribly interested in doing anything
but generating games that the critical apparatus can’t really process
appropriately. That’s also a push in the
right direction: that games should exceed criticism’s grasp encourages both
mediums to excel, and in the case of criticism, that’s particularly necessary.
Astute allegations of sexism and nepotism on both sides
aside, #gamergate brought something to mind that I’d stopped thinking about
long ago: video game criticism is broken.
And while one could say it’s broken for the same reason that poetry
criticism is broken (it’s almost always hinging on professional courtesy
between reviewer and game-maker, and those game-makers often take especially
effusive and talented reviewers on board as writers) I think there’s a more
fundamental challenge facing games criticism, that of an effective critical linguistic
framework. This problem has been around
for a long time, ever since gaming magazines tried to review games with such
categories as “fun factor” in an attempt to represent their value as
experiences, and it’s sustained in the manner in which the apparatus continues
to rely on numeric or graded scores for artistic products, an absolutely absurd
conceit when you consider that the same process is being applied to Call of Duty and Depression Quest. There have
been pushes away from this, often in the form of heated issue pieces, like
Leigh Alexander’s critiques of the representation of women in video games, and
Stephen Totilo’s highly conventional measured journalistic writings which aim
to treat games with the same level of discourse as any other artistic medium,
but the bulk of the industry seems content to avoid any kind of real critical
discussion of games as an art form in favor of utilizing a set of in-house
comparisons and slapdash subjective linguistic frameworks that address
outcomes, rather than processes by which games achieve their goals. Even people like Anita Sarkeesian, whose
remarkable videos have polarized elements of the enthusiast community, rely on
linguistic frameworks that struggle to convey their own points (though, to be
fair to Sarkeesian, a great deal of that can be attributed to the massive scope
of her undertaking, and the breadth of her audience).
When I talk about developing a lexicon for discussing
semiotics in games, I don’t just want to see it applied there: the discussion
could easily escape the medium, and take root in any number of semiotic
dominated fields. Discussing the impact
and evolution of branding, discussing the way that mechanical engineering and
design shift over time, discussing the manner in which signage functions in
different cultures; these are all valid venues for the application of semiotic
discourse, but they’re far more often discussed in terms of their outputs (and
how those outputs might be gamed or manipulated) than their moving parts. I
don’t know how a critical or academic language for semiotics could be
generated, short of a massive undertaking, but when I consider the potential
impact, it seems worthwhile. Games have
long stood as a tremendously influential medium and a remarkable instructive
framework; participatory frameworks are almost always super effective. But without an effective language to outline
the flow and function of semiotic language, we can barely discuss things like
Sean Tejarchi’s LiarTown USA with
more depth than nodding and saying “that’s funny.” Even Jezbel.com,
where feminism goes to die, managed to miss a joke at its own expense,
recommending that people buy a “Social Justice Kittens” poster that lampoons
the very kind of language and infantile discursive discourse that Jezbel relies upon. That says something about the state of
semiotic discourse: we’re capable of engaging with the modes and models it
presents, but when it comes to reflecting on the function of those meanings, or
even just deconstructing them in the broadest senses, we seem to consistently
come up short. It’s just especially
apparent in games, and until we have a language for discussing the frameworks
games rely upon for generating their discourse, we’ll always be discussing it
in childishly broad terms, struggling to express the profundity of what our
experiences have shown us as consumers of this, the most nascent of artistic
mediums.
1 comment:
Post a Comment