Beautifying QR Codes in the Age of GenAI
Raphaël Semeteys
Comment Stable Diffusion et ComfyUI permettent de transformer des QR Codes fonctionnels en expériences visuelles originales tout en conservant leur lisibilité.
Description shown in the presentation language. (French)
If the player is unavailable, use the timestamp links.
Read transcript
- For this short session, we get to enjoy something a little
- fun and a little lighthearted.
- We'll be welcoming Raphaël Semeteys, who will be presenting, among other things,
- Some tips for making our QR codes look better, especially the ones where you
- You can leave feedback or other comments. So it'll be fun.
- Raphaël is a Developer Relations Specialist at Worldline.
- He's an architect; he does all kinds of things.
- He's even a yoga instructor.
- So I'll let you enjoy his little rant about QR codes
- and enter his world.
- I'll hand it over to you, Raphaël. Thank you, thank you.
- Hello, everyone. Thank you for sticking with me this long.
- That's the very end of DevFest Nantes.
- Right before the closing keynote, we're going to talk - thanks
- Ambre for the intro -
- about QR codes and what we can do to make them look better
- in the age of GenAI.
- You introduced me—no need to say anything else except that I'm a geek.
- Yes, I co-host a podcast called "Projets libres" about open source,
- in French. Go check it out—it's great.
- What I suggest is that we start by just reviewing
- a couple of quick basics about QR codes to help you
- understand how it works.
- And then we'll play around with GenAI on that.
- So, QR codes were developed in 1994
- It was developed in Japan by this man, Masahiro Hara, at Toyota,
- and became an ISO standard in 2000
- and we can see that it's still going strong, since last year,
- there had still been an update to the spec.
- It is said to be inspired by the game of Go.
- Actually, it's a 2D version of the barcodes we were already familiar with
- with black squares on a white background or vice versa—it depends
- how we see things.
- It allows you to store digital content, alphanumeric characters,
- binary data, or kanji.
- We can see the Japanese origins of this.
- They come in several sizes, which are defined in the specification or "versions",
- that's the official name in the spec for V1, which is 21 squares by 21,
- Up to the V40, it's 177.
- And in fact, each version is four times larger than the previous one.
- There are also a number of things that are in the spec.
- I'm not going to go into all the details.
- There's one thing that's important: it's a built-in mechanism
- error correction.
- That's what makes it possible to scan QR codes, even when they're
- partially illegible or damaged.
- The Anatomy of a QR Code.
- There are therefore certain requirements that are defined and
- mandatory for
- QR code readers can scan the code.
- There are elements that make it easy to read.
- So, we see this concept of a "white zone" that will
- outline the QR code and the position markers—also known as the "eyes."
- So, those are the three squares you'll find.
- And that will allow a scanner to recognize: "This is a QR code."
- So, I can definitely recognize the QR code.
- There is also an alignment pattern, a
- or more than one, which can be placed within the QR
- code—it's the little eye, if we
- means right there, in the lower right corner.
- And that helps with orientation, since sometimes we're not always
- directly in front of the QR code, so that's how you handle it.
- Depending on the version—and there are several of them—but in this case,
- I'm on version 3, 29 out of 29, so right now, I only need one.
- And all of that is defined in the standard.
- And so the content is stored in binary format.
- A black square is worth 1, and a white square is worth 0.
- The dotted lines you see here help to set
- the size of a module.
- A square is called a "module" in the spec.
- So when the scanner looks at this, it will determine the size of the module.
- And then, we have metadata.
- That's what I highlighted in blue.
- We'll store things in designated areas within the QR code
- such as the format and, consequently, the level of error correction used.
- We'll talk about that.
- I'm not going to talk about the masking patterns or the version itself.
- So, starting from there, beginning at the bottom, right there on the right,
- By moving up in a zigzag pattern, you can start storing the contents.
- So here, I've put a white square, so that's a zero.
- And then there's one.
- And now I'm zigzagging my way back up, one-one, one-one, one-zero.
- There you go.
- So, we fill in all the gray areas like this, then we go back down,
- Then we're done, and we head back up. That's how a QR code works.
- The error-correction mechanism—that's what interests me.
- That, as I said, makes it possible to always display the QR code
- usable, even when it is partially damaged.
- This is done by adding additional data that will compensate for it
- any missing data.
- That will add rows and columns, so it will increase the size, so
- the QR code version, possibly.
- The implementation is the Reed-Solomon algorithm.
- That's common knowledge.
- I'll spare you the details of polynomial algebra
- and finite bodies, but these are mechanisms that are
- used for DVD storage, for one thing, or in RAID 6 configurations,
- on disk security, as well as on network communications
- such as DSL or certain types of satellite communications.
- So, it's not something completely new.
- since QR codes aren't exactly a brand-new concept either.
- Basically, it works with 8-bit codewords that are either codewords of
- data, or correction codewords.
- And then, they will be stored using the mechanisms I described.
- There are four possible levels of correction.
- So, it ranges from low to high.
- And depending on the level, it ranges from about 7% to 30% of the QR
- code that may still be unreadable.
- But with this error-correction mechanism, we'll be able to
- reconstruct the data.
- And that's what has historically been used to customize QR codes.
- Because, actually, I put my company's logo there, so I'm working
- At Worldline, yes, we know that.
- It was placed there, in the middle of the QR code.
- And actually, the scanners will interpret that as an error.
- So, it needs to fall between 7% and 30%, depending on
- of the QR code version.
- That's the trick—it's a hack that lets you always scan the QR code
- code by inserting an image into it. Okay?
- So that's the standard way, so to speak, of generating QR codes.
- So now, what could be more fun than that?
- Because this time, I changed the color and then added a logo.
- What could be more fun than that in the age of image generation
- and therefore AI?
- Part Two.
- What I'm going to do is...
- For those who know me, I come from the world of open source, so to speak
- and I really like doing things locally.
- For me, my use case was: I want to do all of this, run the models,
- run everything on this PC.
- That's another important thing, too.
- What I did was decide to use
- Stable Diffusion, which is a family of image
- generation models.
- I chose a version that isn't very recent because, for my
- use case it's enough.
- It's a smaller model than the more recent ones, such as Flux, etc.,
- which can also be run locally.
- How does Stable Diffusion work?
- It performs progressive noise reduction on an image.
- It takes a noisy image or an image that I provide as input
- and by working in steps, it will progressively remove the noise.
- He reduces the noise by adding elements to the image that will
- follow the prompts.
- This is the text where I'll describe what I want to appear in the image
- and possibly an image, if I provide one as input.
- This is a family of models that are referred to as "open,"
- in the sense that we can retrieve the weights and thus run the model
- locally, and possibly adapt it, fine-tune it, and create derivative versions of it.
- There is a de facto community centered around this openness that
- is quite dynamic.
- There are quite a few components available, including ComfyUI, which is a front-end
- or a GUI that's also installed locally, which will run right
- there on my PC, and
- as we can see, it allows you to create workflows.
- This is the image positioned a the bottom.
- So it's completely no-code.
- You can't tell it's written in Python—everything's fine.
- Because there's a lot going on behind those knots.
- We're going to run some models, we're going to run...
- You'll see some things that we're going to add to our workflows.
- But it's very modular.
- In this case, it really is open source—GPL v3.
- That openness means there's a community that contributes
- a lot of extensions.
- They're called "custom nodes" in ComfyUI, and they have the advantage of
- to also run on Windows, Mac, and Linux.
- For ControlNets, what I need is,
- I need to have a technique because what I'm going to do is,
- It's just that I'm going to use a ControlNet that I've already
- generated, because for generating
- ControlNets, there are plenty of libraries that do that,
- There are plenty of websites that do that, etc.
- So I didn't include that in my workflow—I could have,
- But that's not necessary.
- I want to have a ControlNet as input to modify it using my workflows
- to turn it into something else.
- So what did I say !?
- ControlNet ? I meant QR code.
- And what I'm going to use for that is a technique called
- ControlNet—that's why I slipped up.
- So, what is ControlNet?
- These are additional components that I'll be able to add to my
- workflows, and these correspond to additional AI models that enable
- to add an additional check to the generation process.
- What I need to explain a little is this latent space that you
- may have seen in a previous slide.
- In fact, Stable Diffusion—and that's the original innovation
- of Stable Diffusion is that it doesn't work
- on the pixels and on the image itself; it works on a compressed version
- and represented in a multidimensional vector space,
- kind of like LLMs, in general, with the concept of embeddings and all that,
- We're on the same concepts.
- That's where all that denoising process I told you about will take place, and so on.
- And once the image has been captured, it will then be encoded
- in pixels and displayed.
- And ControlNets—they allow you to influence this
- the mechanism that takes place in that latent space by extracting only certain
- characteristics of an image that I could provide as input.
- And what I want here is a specialized ControlNet
- on: I recognize that this is a QR code, and I'm going to retrieve these
- features to make sure that when you create a new image,
- you transfer the characteristics of the QR code.
- I use a specialized ControlNet called QR Code Monster,
- whose purpose is precisely to do that and ensure that it remains readable,
- Of course, because otherwise, it's not very useful.
- So, what I'm suggesting is that we give it a try.
- What I did was record,
- and I assure you that everything was done on that PC, but I recorded my
- executions—because I want to show you several examples, so I sped them up
- because otherwise, it just doesn't
- hold together at all in 20 minutes.
- Okay? From here on, I'll show you this
- I don't know if you can see much, but in any case, you can see that there are some
- boxes that are linked to one another.
- The green box is the prompt.
- Basically, what I'm saying is that I want a rural landscape with the sky
- blue, flowers, etc.
- The red box represents the negative prompt—what I don't want
- to see in the image.
- So, it allows for more precise control over things.
- The blue section at the bottom contains the items that are
- related to my ControlNet.
- You see, I gave my QR code as an input.
- And here, in the boxes below, I'll choose the version
- the ControlNet I'm going to use.
- This is the version for Stable Diffusion 1.5,
- since that's the model I loaded.
- I can configure a number of things here.
- And then, when it comes to the KSampler, that's really where all the action happens
- mathematics, matrix operations in latent space.
- So, I'm going to do it in 50 steps.
- Guidance refers to the impact of the prompt on the generation.
- And at that point, I didn't speed it up.
- That's so you can see what's on my PC—this PC has an NVIDIA
- 4070 graphics card, and it
- has only 8 GB of VRAM, which is actually pretty low for this kind of thing.
- So here, you can see how long it takes to generate.
- See, there's the seed right there.
- I don't know if you can read this, but that's the random part
- which is used to create the random image
- which is used at the beginning for denoising.
- Okay?
- As you can see, I get this kind of result.
- So, what was I able to achieve?
- With my workflow, I was able to...
- You saw it in the previous workflow; here, I have a few boxes,
- It's pretty simple in that regard, but I already have
- a ControlNet at the bottom, but you see, the more boxes I add, the more I have
- parameters I can adjust.
- So that gives you more precise control, but at the same time, it can be a little...
- It's easy to get lost with all these settings.
- It takes a little fine-tuning to find the right workflow, but once you do,
- Once you have the workflow, you're all set.
- So, we're capable of doing that.
- If I fix the seed—that is, if I say, “I don’t want
- random elements in my workflow".
- There's one thing you need to understand about Stable Diffusion:
- the same settings on the same workflow,
- with the same values, will always generate the same image—every time.
- It's math—there's no magic involved, really.
- It's math—it involves matrix operations performed in latent space.
- So, the guys who made ComfyUI are really smart because they
- said: "We're going to put all of these elements into the metadata."
- Image EXIF data.
- That means an image that was generated using ComfyUI,
- I just drag and drop on the ComfyUI interface, and boom—I've got my workflow
- that made it possible to generate it. Awesome.
- Okay.
- So, if I set the seed, I can, for example, change the prompt.
- Here, I used the same image and the same settings, and here I say:
- I can play around with the style.
- So, I'm going to say: I want it to be a bit like a futuristic building.
- That's what you get.
- On the other hand, it's clear that I'm having a little
- more trouble with the image's content,
- My tango dancers—they're a little stuck in the QR code.
- They're a little weird. We'll deal with this later.
- Let's focus on the style first, and then we'll see what we can do
- on the content and see if we can do things a little better.
- Okay?
- So, when it comes to style, what I can do is have
- some slightly more advanced prompts.
- Same thing here—it's still the same workflow; I just changed the prompt.
- So I went to an LLM and asked it: "Give me a prompt."
- A nice way to depict something a little elven,
- which is kind of, I don't know, I'm getting into the DevFest Nantes spirit.
- No? Well, then, it could take more or less time.
- Because, in fact, in Stable Diffusion,
- there are also models that are capable of understanding text and therefore
- to transform the idea or intention behind the image
- in this latent space.
- We can do that, and we can go even further by
- adding a custom node called
- What's that guy's name?
- ITools Prompt Styler Extra.
- I have to lean over, too.
- If Thierry were here, he'd say, "We're old."
- That's the green box.
- What's interesting here is that it makes it easy to inject
- in the predefined styles prompt.
- You see, these are text files behind the scenes with things that have
- have been predefined and are organized by category.
- We have lots of categories: 3D, crafts, design, drawing, painting, etc.
- And it's very easy to inject a style
- like this in the command prompt.
- So, let's modify the workflow from earlier.
- I just added this box; everything else is already in place.
- Okay?
- So, as I said, there are lots of possible categories here.
- Now, I'm going to choose "Crafts."
- And when it comes to crafts, well, you see, there's a whole list.
- I don't know how many there are anymore.
- That makes 400 different styles.
- And on top of that, the best part is that you can stack them.
- So in this case, I stuck with one style because otherwise, you get stuck.
- There, I set the seed because it wasn't yet, just
- so that we can see the impact.
- And at this point, I start to speed things up because otherwise,
- we're going to wait a little too long. And then, bam!
- In that case, I chose Metal Work.
- This has a bit of a metal vibe.
- There, you can try scanning—it should work.
- I've tested it thoroughly and everything's fine.
- I'm changing my style, so everything else is already set.
- In this case, I chose Crochet.
- Come to think of it, maybe it was my grandmother who had...
- We can do some cool stuff—like wool polo shirts with QR codes. Why not?
- Still in the Craft category, by changing just one setting in my
- workflow—that's what interests me.
- To pay homage to its Japanese origins, I say "origami" because it
- It intrigues me.
- Origami, QR codes—I don't know how that'll turn out.
- So that's how it turns out.
- It feels a bit like paper, but with origami, you're not supposed to cut it.
- So, origami—I don't know.
- And a last one to wrap things up, because I thought this
- one was pretty cool.
- And besides, it'll give me a clue as to the true origin of QR codes.
- Actually, maybe it was the Romans who invented QR codes—maybe.
- Come on, it's okay.
- Here, you can see that we're actually capable of doing some pretty funny things
- and it's relatively easy once you've tweaked your workflow.
- Here, I've added some more, and you can even mix them together.
- Next, you need to have things that are compatible with one another,
- otherwise, the styles are starting to get a little weird.
- Okay?
- That's where you can experiment with different styles—it's cool.
- Now, I'm going to get back to my issue with the tango dancers.
- And besides, as Ambre said, I'm actually a yoga teacher.
- In my case, I needed QR codes for things that had
- connections to yoga.
- What I want to do now is, in the QR code,
- inject a yoga pose.
- Just the posture, by the way.
- I want it to look like a real human and all that, but I want
- that specific posture. To do that, what do I do?
- I also use ControlNets, but this time,
- they will extract other types of information from the image.
- In this case, it's going to be the depth and the edges.
- Why am I doing this?
- Because I only want to focus on posture.
- Then, following the prompt, I'll say: "The guy is dressed like that," etc.
- Okay? So, what's going to happen now?
- It'll just take that and then do it again, with my previous ControlNet
- for the QR code.
- Here, I have three ControlNets; send all of that to the latent space.
- There's a ControlNet called OpenPose that seemed promising,
- but that's not specific enough for a yoga teacher like me, so it won't work.
- After trying them out over and over again, I use these two because
- they can be stacked.
- Here, as you can see, I'm adding the box at the very bottom.
- Here are the nodes I had—the prompts.
- Here is the ControlNet for the QR code.
- And here, at the very bottom, are two columns for extracting the posture.
- Here, I have the first one, where you can actually see the image I use as input
- the second input of my workflow.
- And then I choose the type of ControlNet I want, the type of preprocessor,
- because it involves image processing.
- It will extract that from the image and send it to the ControlNet above.
- On this ControlNet, I can set the force.
- Here, we can tweak things a bit. When does it start?
- When does it end in the generation process?
- And then, I stack them—I put two on top of each other.
- Here, as you can see, the data has been preprocessed and extracted.
- There you go. There's the depth, there are the contours.
- And all of that is fed into the generation process, which I'm speeding up.
- And you end up with something like this.
- What's interesting is that here, it blended the two together a bit.
- You see, it's no longer just an image that's superimposed on top,
- because there's also
- the other ControlNet, which insists it has to be a QR code.
- So, it lets you do some pretty cool things.
- Actually, I think his hair is a little too white.
- I didn't write "don't include gray hair" in the negative prompt.
- That's where it generates that.
- He doesn't have white hair, but he has a few gray hairs at his temples.
- I don't know what I'm trying to do.
- Maybe you've been trying to guess—I'm trying to do
- something that looks just like me.
- Other than that, I really did lie in the prompt, because I turned 40 a long time ago
- but a man who's somewhat mixed-race, etc., wearing glasses, etc.
- It turns out that, after all, I am able to replicate the posture.
- It's still scannable, and it works, but it doesn't really feel like me.
- What can I do to make it really reflect who I am?
- Because I'm a psychopath.
- Okay, I'm going to transfer the face now.
- How do I do that?
- Here, I'm using another technique called Image Prompt Adapter,
- IP Adapter, which allows you to use an image as part of the prompt.
- With models
- All of that comes with pre-trained models, etc.
- There is a type of IP adapter called FaceID, which is specialized
- in facial recognition.
- So, he's going to extract the facial features and incorporate
- them into the new generation.
- Also, by the way, I use a second type of model as well
- called FaceDetailer, which is a post-generation tool and which is
- trained to enhance any face, whatever it may be,
- on the nose, lips, mouth, facial expressions, etc.
- I'm going to combine all of this into my final workflow.
- Those are the parts in yellow that I added there.
- As you can see, we still have the elements from our previous workflow.
- Right here at the top, I have the section on Face ID and the IP Adapter.
- That will be sent, and it will generate an initial image.
- And then the result of this first generation will be sent down,
- in FaceDetailer to generate a second, even better image.
- Okay, so let's get things started.
- And while it's rolling, I zoom in on that part,
- Face ID, just so you can see.
- In this case, it comes with a lot of settings.
- That's what's a little crazy—we actually have a lot of control.
- Here's the image I'm using as the starter.
- In this case, I'll choose one, because there are several
- versions of Face ID, etc.
- So, I'm choosing V2—what's the strength, and then,
- There are also quite a few settings you can tweak here.
- There you go—that starts the generation; the rest is just like before.
- Here, we have the first image that's been generated—it looks just
- like me, in every way
- a little more than a moment ago.
- And here, as you can see, it's been processed through FaceDetailer.
- We're in the second half.
- You see, FaceDetailer is linked to Face ID.
- So, it's going to get better.
- Here, I have a node that comes before and after FaceDetailer.
- If I zoom in, maybe you'll be able to see it.
- That's better. I'm younger.
- AI always makes us look even better.
- I don't know if you've noticed. Anyway, we're capable of doing that.
- That's nice. What did I do?
- There, I managed to transfer the QR code using an initial ControlNet,
- the installation, along with two other ControlNets,
- And then the face—so there, with IP Adapter. That's cool.
- Of course, you can mix all of that together.
- I can come back to the styles.
- Now, if I put the
- I won't go through the demo with you anymore—we won't have time for it anyway,
- But if I put the green box back in, we'll be able to mix
- and therefore to change the styles.
- This is starting to get really fun.
- And then, just to top it off, since I'm even more of a psychopath
- Believe it or not, I said, “I’d like to bring the QR code to life.”
- I want to create animated QR codes.
- Personally, I'd like the face in the QR code to move.
- For that, I use Advanced Live Portrait, which is a specialized model
- in facial expressions.
- He draws them out, knows how to bring them to life, and then chain them together.
- It works something like this.
- That means we have an editor, and it's preconfigured.
- You can make it turn its head, raise its eyebrows—that sort of thing.
- Here, I'll show you another workflow I created.
- I could have chained him to the other one, but I preferred to keep them apart.
- I have an image that I had already generated using my first workflow.
- Here's my expression editor.
- So I tell him which step I want him to take.
- The third node—there's a bit of formalism here—
- I won't go into detail, but it explains how to get there,
- How many installments, how long does it take, etc.
- And then the last node will combine all of that to create
- a video, because a video is a sequence of images.
- So, the frame rate is important—it's the number of frames
- per second that it will use.
- And besides, all of that generates something.
- I'm going to skip that part because we don't have much time left, but at the
- In the end, it looks something like this.
- As you can see, I could tweak it a bit.
- Guys who are good at observing, who have an eagle's eye,
- will see that part of the QR code near the forehead moves.
- I'll have to tweak things a little bit.
- This works; it scans.
- With the iPhone, you can scan.
- But the iPhone sometimes tries to be really clever when it
- He takes photos, and he also shoots videos, etc.
- So it depends on when.
- But if you install an iPhone app, it usually works, too.
- There you go.
- So we're able to do that kind of thing relatively easily
- locally, using open-source tools.
- I think it's fun.
- So I wanted to share this with you.
- Thank you very much.
- That QR code right there wasn't generated using my workflow.
- I could have, but I decided to stick with the one from OpenFeedback.
- So if you haven't scanned any QR codes yet, scan this one.
- The others worked, I swear, but this one did, too.
- That's so you can give feedback. Okay?
- Thank you in advance, and thank you all.
Loading slides…