The Crow

Report: Creative AI Generators (Oct 2022)

Tracy Harwood Blog October 23, 2022 2 Comments

In this month’s special report, we take a look at some of the key challenges in using creative AI generators such as DALL-E, MidJourney, Stable Diffusion and others. Whilst we think they have FANTASTIC potential for creators, not least because they cut down the time in finding some of the creative ideas you want to use, there are some things that are emerging that need to be considered when using them.

Firstly, IP is a massive issue. As noted in this article on Kotaku (Luke Plunkett), the recent rise of AI-created art has brought to the fore some of the moral and legal problems in using it. In terms of the moral issues, some are afraid of a future where entry level art positions are taken over by AI and others see AI-created art as a reflection of what’s already occuring between artists – the influence of style and content… but this is an argument that came to the fore when computers were first used by artists back in the 1960s. Quite frankly we are now seeing some of the most creative work in a generation come to fruitition that just would not have happened without computational assistance. Take a look at the Lumen Prize annual entries, for example, to see what the state of the art is with creative possibilities of AI and other tech. Tracy even directs an Art AI Festival, aiming to showcase some of the latest AIs in creative applications, working in collaboration with one of the world’s leading creative AI curators, Luba Elliott.

As to the legal issues, these are really only just emerging and in a very disjointed and piecemeal way. It was interesting to note that Getty Images notified its contributors in an email (21 Sept 2022) that “Effective immediately, Getty Images will cease to accept all submissions created using AI generative models (e.g., Stable Diffusion, Dall‑E 2, MidJourney, etc.) and prior submissions utilizing such models will be removed.” It went on to state: “There are open questions with respect to the copyright of outputs from these models and there are unaddressed rights issues with respect to the underlying imagery and metadata used to train these models. These changes do not prevent the submission of 3D renders and do not impact the use of digital editing tools (e.g., Photoshop, Illustrator, etc.) with respect to modifying and creating imagery.” This is hot on the heals of a number of developments earlier in the year: in February 2022, the US Copyright Office refused to acknowledge that an AI could hold copyright of its creative endeavour (article here). By September 2022, an artwork created with MidJourney by Jason Allen that won the Colorado State Fair contest was causing a major stir across the art world as to what constitutes art, as outlined in this article (Smithsonian Magazine) and this short news report here –

Of course, the real dilemma is what happens to artists, particularly those at the lower end of the food chain. By way of another example, consider the UK actors’ union Equity’s response to recent proposals by the Government to include a data mining exemption for audio-visual content in its proposed new AI regulation. Why that’s interesting is because already a number of organizations that would otherwise employ these artists, say as graphic designers or concept artists, are rapidly replacing them with AI generated images – Cosmopolitan used its ‘first AI generated cover’ in June 2022 and advertising agencies the world over are doing likewise (Adage article). Some image users have even stated that in future they will ONLY use these tools as image sources, effectively cutting out the middle man, and indeed the originator of the contributory works. So, of course Getty is not going to be happy about this… and neither are the many contributors to their platforms.

And so here is the nub of the problem: in the rush that is now going to follow Getty’s stance (and probably others with similar influence to follow), how will the use of AI generators be policed? This has pretty serious consequences because it has implications for all content including on YouTube, in festivals and contests around the world – how would creative works like The Crow be judged (see our blog post here too)? It certainly places emphasis on the role of metadata and statements of authorship, but it is also as good an argument we can think of for using blockchain too! The Crow for example briefly mentions the AI generator tool it has used, which is freely available to use on Google CoLab here, but it doesn’t show the sources of the underlying training data set used.

AI code source is Pytii Colab Notebook (sportsracer48)

We contend, the only way to police the use of AI generated content is actually by using AI, say by analysing pixel level detail… and that’s because one of Getty’s points is no doubt going to be how their own stock images, even with copyright claims over them, have been used in training data sets. AI simply cuts out the stuff out that it doesn’t want and voila, something useful emerges! So, unless there is greater transparency and disclosure among the creators of AI generators AS A PRIORITY on where images have been scraped from and how they have been used, there is going to be a major problem for all types of content creators, including the machinima and virtual production creator using these tools as a way to infuse new ideas into their creative projects, and as the ability to turn 2D image into 3D object becomes more accessible to a wider range of creators. Watch this space!

In the meantime, we’ll be doing a podcast on the Completely Machinima YouTube channel some of the best creative ideas we’ve seen next month so do look out for that too.

We’d love to hear your views on this topic, so do drop them into the comments.

btw, our featured image was created in MidJourney using the prompt: ‘Diary of a Camper made in Quake engine’, by @tgharwood

Projects Update (Oct 2022)

Tracy Harwood Blog October 10, 2022 1 Comment

This week’s Projects Update on machinima, virtual production and content creation:

The Crow

One of the most interesting creative projects we’ve seen so far using MidJourney, a creative AI generator is the The Crow (by Glenn Marshall Neural Art). Here the generator has been used to recreate a version of the ballet performance portrayed in the short film, Painted (by Duncan McDowall and Dorotea Saykaly). Stunning to say the least and we recommend you play it at least once side-by-side against the original performance too for added insight.

We’re so impressed with the potential of AI generators, whether that’s DALL-E, MidJourney, Stable Diffusion or any of the others that are now emerging, that we’re going to dedicate a special episode of the podcast to the subject next month, so watch out for that!

Jim Henson Company

Jim Henson Company is using real-time animation on their new show, Earth to Ned. Characters are created with Unreal (its the AI in the background) but JHC has been so impressed with the workflow and no post production requirement that it is looking to use the virtual production method more. What’s interesting is the level of feedback in the process that guests experience – they are not aware of the puppeteering in the background, just the virtual actor on the screen, performing naturalistically in real-time! We’ve not seen much of this kind of machinima before although actually Hugh Hancock did some very early work on this and of course Rooster Teeth have done live performances using similar techniques. We can certainly expect to see a lot more of it, particularly for interactive theatre, VR and AR.

Half Life 3

Half Life 3 was never going to be like the originals? This article on Tech Radar is interesting: the author (Phil Iwanuik) contends the Half Life franchise remakes would never be like the originals because the extreme attention to the world of HL created so much pressure the Valve team could never live up to it. We’re not sure about that, but it’s an interesting idea.

source: Valve

Dune: Awakening

A very impressive MMO has launched using the Dune world, currently in beta, Dune: Awakening. Here’s the trailer – we’re looking forward to seeing machinima made with this –

Dungeons & Dragons?

What does Dungeons and Dragons, typically a game played around a table, have to do with machinima? There’s been a rise in popularity of web based shows where people play the game and act out scenes. This group (Corridor Crew) is using Unreal Engine 5 for virtual production (not quite The Mandalorian but sort of similar) to put their actors, real-time, into the environments of their adventure. Check it out here –