S13E01 Zach Daniel - Creator of Igniter & Ash Framework (Audio)
[00:00:00] Intro: Welcome to Elixir Wizards, a podcast brought to you by SmartLogic, a custom web and mobile development shop. This is Season 13, The Creator's Lab, where we're talking to innovators who are expanding what's possible with Elixir, from groundbreaking projects to tools shaping the future with our favorite technology.
[00:00:29] It may seem like magic, but it's really the result of talent and lots and lots of hard work. 
[00:01:27] Owen: Hey everyone, I'm Owen Bickford, software developer at SmartLogic.
[00:01:33] Charles: And I'm Charles Suggs, software developer at SmartLogic. And we're your hosts for today's episode, the first episode of season 13. We're joined by Zach Daniel, creator of Igniter and the Ash framework and principal platform engineer at Alembic. In this episode, we're discussing Zach's latest project, Igniter, a code generation and project patching framework for Elixir.
[00:01:54] Owen: So welcome, Zach.
[00:01:57] Zach: Hey, thanks for having me. I'm glad to be here. 
[00:02:00] Owen: welcome to the party. This is the first episode of Season 13, like Charles said. I'd like to officially welcome Charles as our new co host as well. 
[00:02:09] Charles: Thanks.
[00:02:09] Owen: yay, we're like growing the team here. That's always good and fun times.so to kick off the season, you know, the theme of the season is Creators Lab, we're talking to developers who are maintaining either a project or maybe an entire business.
[00:02:27] Elixir is always somewhere in the mix, pardon the pun. Today we're focusing on Igniter, which is a really cool project.I think this is your first time on the podcast, is that right, Zach?
[00:02:37] Zach: This is yep. It's my first time.
[00:02:40] Owen: Cool, so I guess the first thing to do would be a quick personal introduction, uh, whatever that might mean to you. 
[00:02:48] Zach: yeah. my name is Zach Daniel. if you've been around the Elixir community, you may have seen me in the past at, you know, ElixirConf or I've been on a couple other podcasts, my, Claim to fame, I guess, is, creating the Ash framework. more recently, I'm working on a project called Igniter, in addition to Ash framework, not to scare any of our users.
[00:03:07] I'm working on a complementary project called Igniter, which is a tool for building, code generators, project patchers, and installers with Elixir. and I've been doing Elixir for, I guess, 10 years now, almost? Loving every second of it, not looking for anything else with my life, personally. I think it's a ride or die for me and Elixir at this point.
[00:03:30] that's me.
[00:03:32] Owen: So you're one of those, the elite few who can apply for that junior software engineer position who needs 10 years of experience with Elixir. 
[00:03:39] Zach: Mm hmm, yep.
[00:03:40] Owen: Awesome. So I guess at a high level, tell us a little bit about Igniter. What's the problem that it's trying to solve? How it integrates with Ash, potentially. 
[00:03:49] Zach: In a way, you know, Igniter, it really has nothing to do with Ash insofar as it's a project that's just generally useful for any, anybody who's writing, an Elixir library or package or framework, or even those who are just working on, you can use this outside of the context of a library.
[00:04:06] so primarily what it is, is a tool for, code generation. And I would say sort of intelligent code generation. So the problem we're trying to solve is essentially I guess you could say there's two problems, right? One is installing packages, right? So right now, one of the things that you do is you can use mix new to create a new Elixir project or you can use like somebody else's mix, You know, uh, mixphoenix.new, mixnerves.New, mix whatever to create a project. and that's really the only thing that, that's really the only way to get started with a new project. After that, maybe one of the packages that you install, like by manually editing, you go and edit your dependencies, you add something in. Then maybe there's like a mix task to get started to like generate some code.
[00:05:00] So like with Credo, you can generate a Credo config and with, you know, it's yada, yada, yada. They all have these little tasks, right? So we want to solve the ergonomics of that from, you know, Which is, I think actually quite easy the way that we've done it. so with, when you're using Igniter, you can just say mixIgniter.Install and some package.
[00:05:18] And it goes and like looks up what the latest packages on hex and adds it to your mix file directly. Kind of like if you were using NPM and you installed to like save.but then it also looks to see if there's a mix task. And this is one of the wonderful things, like Elixir has standardized so many things that actually make this sort of thing possible.
[00:05:36] That like our build tool called mix that executes everything, there's only one, I don't have to like, you know, it's really nice. So we look for a mix task called yourpackage.Install. And if it is there, and actually it has to be an Igniter mix task for this to work. But, if it's there we run it and we'll install your package. 
[00:05:55] So, for me as somebody who builds Elixir packages, the DX here is just like wild, because I can just give somebody a one liner that not only installs my package, but also like every, like it changes your files, it sets the project up everywhere it needs to go, it sets the configuration, it does all these different things, and it just does that in one go, and it does it, it shows you a diff, hey, these are the changes that we'd like to make to your project, is that okay?
[00:06:19] So that's kind of the solve for installing packages, right? To just solve this sort of manual process of looking up the latest version or then going to read the Getting Started Guide to see what, what do I have to do to set this thing up? you can just push button install things now. The other thing is making smarter generators.
[00:06:40] right now, the sort of state of the art, You know, without Igniter for writing generators is to use things like mix. I think there's, it's called like mix.generator or whatever that like lets you copy a template from some file. That's like an ex template and put it somewhere in the directory or in the user's project.
[00:06:58] and it's all effectively text operations, right? Like what you're, what you can create files, you can copy files and you can Edit the text of files using things like file. read and like all this like very manual So you just don't do that. You don't do smart code patching in Elixir right now. And so we're building on top of tools. This is not like I didn't invent every concept that Igniter is built on top of right so Sourcerer exists from Lucas, and I forget his last name.
[00:07:28] and we have a project called Rewrite that we use as well for managing like a bunch of staged changes to the file system and all this sort of stuff.but what we do is we have a framework of high level tools that allow you to patch source code directly, right? So you don't think in terms of modifying files.
[00:07:47] You think in terms of go edit this module or any module that calls use Phoenix.component for example. You can do that and then you can say go to the place where it calls this function and change the function name to this, right? So it's these really intelligent tools that allow you to make,sort of scripted changes to files and stage them up for one big change.
[00:08:07] it's really, it's quite ergonomic to use. from the end user's perspective, and it, there's a lot of really, like, a lot of niceties in that process. And one way you can actually try this out right now, at least as of, I think, the recording of this podcast, this is still the case. When you generate a new Phoenix project, right now, if you just do mixphoenix.new it generates a project with a warning, like a compilation warning from GetText. Because there was an update to GetText. That like you have to use a different, instead of using the get text module, if you use like your get text and say backend or whatever. so just as a demonstration for how nice Igniter is to use, we made a mixed task called mix Igniter dot update get text, and we ship it with Igniter.
[00:08:49] So you can add the Igniter dependency to your project, or you can say mix Igniter dot new dash dash with phoenix dot new to create an Igniter project with, that starts with Phoenix's generator. And then you can say mix Igniter. update getText. it'll find all the places that have to update.
[00:09:05] It'll show you a nice little diff with you know, green plus lines. Here's what we're adding and here's what we're removing. And it's a, it's really nice. So those are the two main things that we use that Igniter is for, is for that installation process and then being able to write smarter code generators and project updaters, that sort of thing.
[00:09:25] Owen: Nice. Real quick, just, you mentioned Sourcerer, so we got to mention, Friend of the Pod, Lucas San Roman, the author and maintainer, if I remember correctly, of Sourcerer. We'll paste a link both to Sourcerer docs as well as the episode if you want to hear more context about what Sourcerer does and what it's capable of. 
[00:09:45] Zach: Sourcerer is the magic that, that makes Igniter work, and where Igniter fits in, in the, that sort of ecosystem is that Sourcerer is a very low level, it provides very low level tools that have to exist, right? very important low level tools for manipulating source code.
[00:10:01] So, you use a data structure called a zipper, and you like, you, you're operating on a tree, like the AST, the Abstract Syntax Tree of Elixir. And you can do things like travel up and down and go to the right and left of the abstract syntax tree. and that's really powerful, but it is very difficult to turn those little things into like some sort of semantic operation.
[00:10:27] Like, here's the change I want to make to a file. And so Igniter has those high level operations. for example, we have a set of Of, uh, functions that are in the Igniter. project namespace that are like sweeping changes to a project or not sweeping changes, just sort of project level changes. So you can say Igniter. project. depth. adddependency. And like that knows how to modify the mix. x's file. You can say Igniter, project config, configure, and give it like an app name, a path to the key that you're trying to configure and a value. And it knows how to patch the config files of a project to set that configuration, which is a bunch of different things that you do with the zipper. Like you to go find and figure out what needs to update, what needs to change, all this sort of stuff. So yeah, Sourcerer's really, it wouldn't work without Sourcerer. Sourcerer's the requisite underpinning and Igniter is like. Making that accessible to developers who, you know, know what they want to change about a project, but don't have the potential, you know, the time or the energy to go figure out how to use zippers. 
[00:11:30] Charles: I have a small question going back just a little bit. you were talking about using Igniter to install packages, and I think in your blog post from February announcing Igniter, it mentions with mixIgniter. install that it will check for an appropriate version dependency from hex, and I'm curious, are there cases where that's not the newest version of the package you're installing? 
[00:11:51] Zach: so no, there, there really aren't any cases like the way that when we say appropriate, the only reason is because it's a little bit more complicated to explain, like we don't pin you to the latest version.if a package is on 3. 2. 1, then we put like squiggly arrow 3. 0, you know, or we'll say if it's on 0.
[00:12:10] 1. 7, then squiggly arrow 0. 1, you know, so we just, we put in like a, the least restrictive that we can. but we do also have the ability, it's, this is really useful too. You can mix Igniter. install, put in a package name like Ash, and then you do a colon, and then you can. Customize where it's installed from.
[00:12:31] So you can specify a specific version or you can specify, or sorry, you don't use colon, you use at. So you'd say Igniter at or ash@github colon, the github repository or @path and then point it at your local path. Right? So it can be really useful, kind of like a little global installer, no matter where the packages are.
[00:12:52] and you can say like dash only dev, right, to only install it in dev and all this sort of, we've recently really, enhanced that interface to, to make it more than just installing a package from hex.
[00:13:04] Owen: It's funny, as I'm kind of like processing what you're talking about, Got so much to discuss today. I don't know how, we need more hours, but, I think the first thing I want to talk about is code generation. So in the Elixir space, at least, there's, we're talking about one form of code generation, which is run a task, which inserts, like maybe creates or like injects code into some files.
[00:13:28] That you'll see in your repository. And the other type of code generation that comes to mind is macros, which Ash is definitely macro heavy. We could talk about DSLs. That's probably another couple of hours of conversation. so yeah, like. Going back a year, I think I started experimenting a little bit with, mixed tasks to, we're both coming from agency work, right?
[00:13:53] Like you've got, sometimes you need to start up a new project or you're doing some maintenance on a project and you need to, you know, add a dependency. And then there's all the setup that you have to do in order for that dependency to actually do anything useful. And so what I really appreciate and something I've tried to tackle just using regular mix tasks.
[00:14:11] is, making it easier for developers, especially our junior developers to, to add features to add, dependencies and scaffolding and boilerplate code really quickly so that they're not spending a day or a week on like really tedious stuff that could be automated, you know, like what's valuable to the client is not like We added a dependency to your app.
[00:14:34] It's, we added some really cool features that your users can actually interact with. And so can you talk a little bit more about kind of like the impetus for starting Igniter in the first place?
[00:14:44] Zach: Yeah, so, fun fact. I mean, well, it's not really in my case related to the agency work. So, well, it's complicated. There's, there's multiple axes, I guess, here of like benefit, right? But the real, impetus was people kept asking me for generators, and I just wasn't willing to write a generator that used the tools that we had.
[00:15:09] And so we, Ash had like four years, no, no generators, right? Like we had like hundreds of people in production and there was not a single, there was no ash. new mix task. There was no, new resource generator. and so a lot of it kind of comes from the idea that like, okay, if I'm going to do this, I'm going to do it right.
[00:15:26] I'm going to write it in such a way that I can do smarter things in my installers and I don't have the same problems that I would have otherwise. I, what I really do not want, like I think it's a major issue. So I love Phoenix, right?, and Ash is not, for anybody listening who's heard that I work on Ash Framework but doesn't know what it is, that's not an alternative to Phoenix.
[00:15:46] It's something that you would use in addition to Phoenix if you were wanting to build a web application. However, for new users, the decision point at the front, when you have to generate a project and you have to decide, you can say dash dash no and insert 10 or 15 things here that you do or don't want.
[00:16:01] That's a big problem for me in terms of the new user onboarding experience. So Igniter flips the, flips the script on it by being able to write smart Igniters, I want to, and I'm going to be pushing for this over the next year or so. As I, as Igniter gets better, we have a bunch of Phoenix specific tooling in Igniter now to 
[00:16:18] add a scope to the router, to patch this kind of file, to like, add a plug to a pipeline, all this sort of stuff, right? it's my goal to get the, to get Igniter into Phoenix and to rewrite the Phoenix New experience such that Mix Phoenix New gives you nothing. And then you have to opt, you can progressively add pieces as you need them.
[00:16:41] So you're like mixphoenix. new gives you like almost a bare bones Elixir project. And then you can say mix, you know, mixphoenix. add or install liveview or mixphoenix, something like that, right? To progressively build up. And of course, we need that like one bit. That one magic command. So maybe mixphoenix.
[00:16:57] new does start off, gives you everything, but you can just say like mixphoenix. new dash dash bare or something like that to start you with an empty project so you can build up over time. so that's, that was the big impetus, but you're absolutely right that We're almost, so we're almost done with the, Ash authentication Igniter installers.
[00:17:16] and that is basically like kind of, I think, a better middle ground between PhoenixGen Auth. Where you generate all of the code. It's we are going to be generating some code that still calls into an authentication library so we can patch security vulnerabilities and all that sort of stuff.
[00:17:31] but when our clients come to us, and this is a really, your point is very salient when our clients come to us and they're like, okay, we want to start an app, they don't really like that. That first like week or two is really important, right? Especially if it's like Greenfield new work. And if we have to say things like, okay, we're working on letting users sign in. Or like, we're working on, you know, setting up the structure of the app.
[00:17:55] You know what I mean? that's not impressive. To them at all. 
[00:17:59] Owen: 100%.
[00:18:00] Zach: Yeah. They don't understand. And they should, nor should they have to understand that like, Oh, well, you know, signing in is complicated. There's a lot of security things that we have to solve here. And it's none of that crap matters to them.
[00:18:10] Charles: But it already exists in other apps. Why, why does it take so long? Yeah.
[00:18:14] Zach: Everybody's app can do sign in. Surely it doesn't take everybody two weeks or a week to set up authentication. Right? so there is a big part of that in terms of Like the getting started for Ash now is really nice and especially once the Igniter, the Ash authentication stuff is done, there's a one liner command because you can chain installers.
[00:18:32] That's the other thing with Igniter. They're composable, which is very important because we can run them in any order and we can stack them up and we can make one big diff. So there's going to be a one line command. That could get you a full featured Phoenix application with a UI that lets users sign in and sign up, that also has tooling for GraphQL and JSON API and Ash Postgres all installed, ready to go to just, you could start developing your first feature like 10 minutes into the first generation of your project.
[00:19:01] So that is big for an agency. Yeah.
[00:19:03] Owen: you've said another magic word here. Authentication. This is I'm just, I'm so glad this is our first episode. So like, yeah, I've, so another place that I've added some generators is a side project, open source project for Elixir that I have called WebAuthnComponents. I know we talked about this a couple of years ago at ElixirConf, if I remember correctly, maybe three years ago. Time is a flat circle. So, uh, So yeah, and it took inspiration from the Phoenix auth generator, you know, I'm taking the approach, of, you know, I've got some components that are going to wrap a lot of functionality for you so that you don't have to know the entire ceremony of web authentication and, registering and authenticating passkeys in your Phoenix app, but you still want to take ownership of like the templates, the LiveView, like you might want to customize the error messages that are being shown to the user and that kind of thing.
[00:20:00] So, so yeah, finding the right balance is something I was reading in your article, which we'll have a link to in the episode description. there's always a balance of like, how much Do you abstract away in the package or library code and how much do you want to generate into the developer's applications so that they actually can modify it to their heart's content?
[00:20:24] part of the impetus there is also, you know, like it's kind of following the typical context, you know, it kind of looks like a normal context if you're looking at you know, the schemas. The things you can do to the schema through the context. But you know, lately I'm getting a little bit more,enthusiastic about event sourcing in CQRS.
[00:20:45] So like with this approach, I can go in and rip out all the stuff that doesn't fit the patterns of my app and then start making changes there.using Sourcerer and I'm really interested after we talked at this year's ElixirConf, I'm interested in trying out like a branch of, rewriting things with Igniter. And I think a couple of questions that come up there as I'm reading through your article is, so it looks like obviously we can modify and insert Elixir code using Igniter. Does it also extend to files like JavaScript or like maybe you want to generate some GitHub action code, that kind of thing? Is it, does it go beyond Elixir?
[00:21:25] Zach: It does go beyond Elixir, but only barely. So like, the magic only works for Elixir. Because like, inside of Elixir, we can do things like use Sourcerer to parse Elixir code and modify it, right? But, you absolutely can stage a change to a Javascript file if you want. but it's just a text. That is just text, right?
[00:21:47] There is nothing that would like parse JavaScript and then give you a zipper over JavaScript or something like that. I actually do want that to be a thing at some point. It's gonna have to be its own, there's no such thing as, you know, A shared set of operations over an abstract syntax tree of two different programming languages.
[00:22:06] Cause like defining, like they fully different structured and all that sort of stuff. However, I think JavaScript would probably be like the first target there, especially like we already run into that with, when you install it with Phoenix, it's you want to modify the tailwind config and that's a JavaScript file.
[00:22:20] And we can do some reg XE crap to go find like the place and but it's not worth it with Igniter. You can just say, add a notice. That runs when the installer is done, that tells the user to go do something. and that's what we do because it's not really worth it to, you know, slice and dice up a JavaScript file.
[00:22:36] But I would really like it at some point, if we, it just requires somebody to write a JavaScript parser in Elixir, that's like stable enough to be used for that case. So it may never happen, but, but yes, you absolutely can modify regular, any tech, any file on the file system. 
[00:22:53] Owen: I'm looking at, so just to give our listeners a concrete example here, so using the WebAuthnComponents example.you know, there's a lot of things that happen in JavaScript as just a hard requirement because this is a browser technology that we're interacting with through PhoenixLive components.JavaScript hooks.
[00:23:12] In Phoenix terms.so A, I'm just injecting a bunch of, or like a handful of JavaScript files. And then as I'm looking at my actual source code here, I'm actually using some string replace and some Regex to, to inject the actual hooks into the app. js file for whichever application that you might be injecting this into.
[00:23:34] It could get messy if like things are just not in the right shape.so I guess I'm kind of curious, does Igniter allow. As an escape hatch, does it allow you to also do some Regex? 
[00:23:44] Zach: yeah, 
[00:23:44] so what you can do is you can just say update file and we give you, you just give us a function of the rewrite source, which has the textual contents, and then you can just set the new text contents of the file. what is nice about the way this framework, because it is a framework, I know people, you know, I guess I'm a framework writer.
[00:24:02] The reason it's a framework is because it, you like give it, here's the files I want to change, like go change this file. And like, ultimately it's something at the end that actually does this. and right now it does a relatively, I don't want to call it that naive, but it just displays, here's all the files we want to change.
[00:24:19] Here's, you know, and one of our, one of our users just now, just the other day was like, Hey, I'm trying to change like 80 files. So like this makes a massive difference, right? But that's going to get richer over time. And so what we want to do is make that actually interactive. Where instead of just saying, yes, no, do you want to apply these changes?
[00:24:37] There's actually going to be tools to, if there's a lot of changes, we'll let you page through them to see like, okay, show me the changes in like a, in an easy to read way. And also to reject individual changes. Right? So that you can say, actually, don't make that change to that file. And, as we go on that path, it actually makes it a little bit less dangerous to have, one big task that does a bunch of stuff, and then maybe you get the app. js file wrong because they've done something weird, which they probably know about. That's the other thing, this is a best effort thing, like, you can't fix every, you can't do this for everything, so you just have to say, Hey, we couldn't make this change to this file, or you have to do your best and they'll fix it.
[00:25:17] so that's probably what that would look like. And then if, yeah, if you got it wrong, they'll just say, do everything except the changes to this file and then write it.so yeah, that's the nice thing about this approach is we can grow the user experience over time and those generators don't have to change anything.
[00:25:30] They just use our tools to modify the file system.
[00:25:34] Owen: Nice. 
[00:25:35] Charles: did you have any surprises as you were building Igniter?
[00:25:38] Zach: I, yeah, I actually had a really big surprise. There was a part that I thought would be like. Table stakes, basic stuff when I started the project and it turned out to be, it still is the most complicated part of the project. And it's not the source code patching or anything like that. what it is, is when you say mixIgniter.
[00:25:59] install, right? What do we have to do? So first we check hex, right? And then we're like, okay, we find it. We add it to the mix. exe file, but then we're going to go look for a mix task defined by that project. But that means compiling that dependency. And, when you, the way dependencies work in Elixir, like, just in general, right, but it's like, you can bring in a dependency, and one of your other dependencies has a optional dependency on the new thing that you brought in.
[00:26:29] Right? And so the way optional dependencies work is like when it doesn't depend on that thing, but when you do bring it in, it has to recompile because it's going to, it's using things like if that module is compiled from that dependency , that's optional, then I'm going to be compiled differently and do different things and et cetera, et cetera.
[00:26:46] So we can't just sort of try to compile the dependency in isolation. We have to recompile the whole project. And we have to, not only do we have to recompile the whole project, we have to like tell mix to re evaluate the dependencies and recompile the project. And that actually flat out, there is no way to do that.
[00:27:03] That does not work. And this is a problem for language servers as well. the language servers that are out there right now kind of fall flat when you make dependency changes. Like they all, like I'm pretty sure. Sorry, I know there's three at the moment. We'll be one at some point very soon, I hope. but as far as I've seen, they all can crash and get weird when you add a new dependency to your project, and so you're typically like deleting its folder and restarting your language server or whatever. we do this totally wacky, crazy thing that fakes it. we like pop off the mix context, like your project.
[00:27:35] And then we just like re evaluate your mix. x's file, which says, okay, well, that's the mix project now. And then we like, we, I hacked my own copy of the mix deps compile task that like loops over dependencies and make some like weird choices to avoid crashing. When we, it's, it was, and it just was really hard.
[00:27:53] And I really thought that, I thought that there was going to be like a mix deps. recompile task that would just like do this for me. And I was very sorely mistaken. and I've, Jose has talked about this a little bit, even that this may exist, this may have to exist for the sake of language servers over time.
[00:28:11] And so I'm hoping that when that happens, I can crib off of that and we, Igniter won't have that weirdness in it. because it's just, it's a nightmare. 
[00:28:19] Charles: Sounds like that was a bear to work through.
[00:28:23] Zach: Yes. Yeah. And I was just so upset because. I had solved all my problems and I was ready to work on the fun stuff. And it was like a week and a half of like this crap. And I kept thinking it was working, but it wasn't, it didn't like, I would go, okay, now I'm going to solve this package. And it didn't work for that package.
[00:28:37] it was just, it was one of those projects that just, you know, you're like, I'll just blast this out real quick. Then I can do the fun stuff. And then two weeks later, I'm just like pulling my hair out that I don't have to, yeah, it was rough.
[00:28:49] Charles: That's when it hits.
[00:28:51] Owen: I know, right? Yep. 
[00:28:54] Charles: Uh huh.
[00:28:58] Owen: Charles, you'll be, you'll be, you'll join the club someday.
[00:29:00] Charles: It's, oh, it's started.
[00:29:04] Zach: One of us, one of
[00:29:05] Owen: Right. So I'm curious, did, was this inspired, have you seen examples of similar tooling from other languages or was there something else? mean, Sourcerer obviously is kind of like you're building on top of Sourcerer. was there prior art or some other inspiration that you were taking to build Igniter?
[00:29:25] Zach: Yeah, so I had the general idea of like smarter generators, that are patching projects. Was something that I had thought of a long time ago when we first started with Ash, because Ash DSLs are very regular and so they actually lend themselves really well to being patched and upgraded and stuff like that.
[00:29:44] And that's actually proven to be true. That was a supposition I used to have and it's proven to be true with the way our Igniter installers work, which is, I can like It's kind of like how an Ectoschema, you know, that,the Syntax of an Ectoschema ought to be pretty patchable. Add a new field, right?
[00:29:59] It's not, you don't generally have like conditional, if this, then add a field to my Ectoschema, it just doesn't look that way. It's more like it's data, you know, pretending to be code or code pretending to be data, whatever it is. Um, and, but then,Surface came up and, Marlus had done the, like these really cool installers that leveraged Sourcerer.
[00:30:20] Partially. Also some Regexes and some, some goofy stuff that I didn't like. But you know, of course this is like, he was bleeding edge at the time, right? So there's no, no judgment there. But when I sat down, I'm like, no Regexes, never, not a Regex ever. that's a rule in Igniter. There's no, there's not a Regex that we use.
[00:30:36] and, So that was, I think, one of the original impetuses of like the design pattern or the inspirations for the design patterns and leveraging the AST to do this. Not not what I was thinking at the very, you know, long in the past of because the text is more regular, it'll be better, easier to parse or something like that.
[00:30:53] But no, it's just Sourcerer that makes that magic work. 
[00:30:56] Owen: On that note is, so I'm, I'm seeing that with Elixir 1. 18, which is currently in development, there's, I think this is new. This might have been like stealth, stealthily introduced in the current version of Elixir, but there's a new mix format dash dash migrate feature for, and I forget what that related to and their change log, but,
[00:31:18] Zach: It's for unless, they're deprecating unless Yeah 
[00:31:22] Owen: so - unless - the keyword, you know, the anti if statement is being deprecated. And so when you run this mix format dash dash migrate, command, it will replace your "unless" uses in your code with "if else" or maybe just "if and," have you looked into, have you looked under the hood of that? Do you know if there's anything in common with what 
[00:31:45] Zach: Briefly. I don't think so. And I mean, the reason is because like, you know, the Elixir formatter doesn't depend on something like Sourcerer. It's like already an AST based formatter doing its own thing. And so I think it's like, The way that it's like a, when they encounter the AST that matches a given thing, like I looked at it, you know, it's, it's a relatively straightforward.
[00:32:07] I mean, the formatter is awesome. Right. But this particular feature edition is relatively straightforward of okay, modify the AST to, when you're at that point. So they're like in a transformation of it and then they can do their work. Um, and I love it. I mean, that's exactly the kind of thing, like, I don't, I don't I think Igniter is going to, I don't think that the tooling for this that lives in Elixir core will ever approach the level of tooling that you can build with something like Igniter because we can move, way faster and we can add a bunch of these, like, high level tools.
[00:32:40] Like, we have basically a standard library of code modifications. but that developer experience. is huge, right? Like that's, we're on that like higher plane of, of developer experience now that I think that we're operating on.
[00:32:54] It's not just like, , are my error messages good? We're at like that second order developer experience, type of thing. that's a great example of it. And I'm really, I was pleased to see it.
[00:33:03] Charles: You, you seem pretty excited about Igniter. Was there a spark, no pun intended, that, uh, uh, really got you to be excited to work on it? 
[00:33:16] Zach: you know, it's funny. I, like I'd mentioned, I was kind of backed into working on it just because I knew everybody wanted generators and it was a good idea. Like they were right. especially for the installers. They're so nice. It's wild. Like when I want somebody to install, when I want a reproduction from somebody, I'm like, Hey, can you reproduce this bug in an app?
[00:33:32] I just type. The command to make a new Elixir project that has Phoenix and Ash and the package that they have a bug with installed. I say, use this to make it your project and copy your resource into that project and put it on GitHub, right? So like just for me, like it has tangible benefits to my workflows for supporting users of Ash.
[00:33:50] But the real thing that has me excited about it is, Ash has always been relatively controversial, right? there's some people who love it. There's some people who think it's the worst thing that's ever happened to software, right? And that's fine. I can live with that. It's not a big deal. I'm confident enough in the value that we're adding and what we're building, but people who use Igniter, everybody likes it.
[00:34:13] And so that's really fun for me. There's it's better than what we have. And it allows you to do new and better things with a better UX. So our users are happy because Igniter exists. There, there's, it's obviously better than the alternative. And it just feels good to use. It has these nice little UX touches, you know, like error messages come at the end printed in red and the diffs displaying, you know, yes, no, accept.
[00:34:36] That kind of thing just feels like, oh, this is like, Well thought out and way more ergonomic than the alternatives. So for me, that's what I like about it. I like it like being in this unambiguously unambiguous area of people just like this thing. Like it's, you know, nobody's going to, I don't think we're really going to get any argument about whether or not this is like a better experience for users or a better way to write generators.
[00:34:58] It's just not what I see. 
[00:34:59] Owen: And it sounds like you're starting to get a little bit of buy in maybe from some library authors and maintainers. You mentioned Phoenix earlier. I think I remember those conversations are starting to happen. I'm kind of curious with let's say you mentioned earlier, like, MixPhoenixNew dash dash all the different flags you would want to opt out of. if we just pick one of those, like the mailer, let's say I start a Phoenix app and today I don't need email, but in six weeks I will discover that I need email for some reason.would it be on Phoenix to add Igniter support for adding the mailer code, or would it be on the, the Swoosh library for doing that kind of scaffolding work?
[00:35:41] Zach: What I would say, the Igniter, like the philosophy that we're building up around it, is that the generators belong in the project, whatever thing you're generating.but those lines are not always super clear cut, right? So, an example of this is like, Everything in Igniter is you take an Igniter, you return an Igniter.
[00:35:59] That Igniter might have modified some files. , it's like a plug or like a con, you know, so it might've staged some changes or it might've done whatever, right? So you can also just write functions that do exactly that. And that's what we distribute. So Ash has, some modules like ash.
[00:36:12] resource. Igniter, which are like, you know, functions that can modify a resource. And in Igniter itself, not in Phoenix, but in Igniter, we have functions for modifying Phoenix projects, right? It's really up to them. I would say if you can put it in Swoosh, then that's a good idea. however, it depends.
[00:36:28] It just depends on how Phoenix is specific, like that in that particular installation is probably not Phoenix specific at all. And so it should just go in Swoosh, but you know, 
[00:36:38] Owen: and you're, if I remember correctly, you're using the builder pattern. So you're saying like it takes an Igniter, returns an Igniter. So I'm thinking through this in real time here, maybe swoosh,does it's you know, basic config injection. Maybe there's a couple more modules that get injected.
[00:36:53] Then like it returns the Igniter. Then Phoenix takes that Igniter and builds in like, all right, here's our email templates, here's a couple more like tests and things that would get written on top of that, and then they're kind of like. Laying on top of each other. 
[00:37:07] Zach: and even Swoosh can have like functions, like create email templates. That Phoenix calls. And so that, you know, like Swoosh is in charge of like where it happens. And so that's how, what we do right now, for that sort of thing. And what's really cool is the way that you can compose these things.
[00:37:22] So every Igniter task. if you had mix swoosh dot generate email template or something, right? Every Igniter task can be called as a task, just as the mix task, or it can be composed using Igniter dot compose task. So when you have like your installer could be packaged up as some part of a bigger installer to fully transparently to you and you don't need to change how it works at all.
[00:37:47] , but then we encourage building up a nice library of useful generators. and making your installer or other things like that, built up off of those smaller, more useful generators. So I think that's a great example where that would be really powerful. And what would come out of it is that when somebody else wants to write a thing that like needs some Swoosh email templates, they don't have to like copy the rules of what that Swoosh has for where email templates go or what they look like, or any of that sort of thing.
[00:38:17] They can just, so it's that collaborative. Code generation concept. That's really just not realistic or possible without a level of standards around what that actually looks like. 
[00:38:26] Owen: Yeah. So Igniter is an open source project. I'm kind of curious what kind of contributions you're getting currently, what you would look for from the community. If we could bring our listeners to the, to the repo, are you looking for issues, PRs, or is it a little bit more closed off, 
[00:38:46] Zach: yeah, I mean, it's definitely not, you know, closed off in any sort of meaningful way. what we really need more than anything right now is for more library authors to write some installers. We have them , Jonathan Manchin does OIDCC the.
[00:39:01] Authentication, like OpenID Connect, he's writing an installer for that, that uses like the new Phoenix features we have and all these other things to write Phoenix specific installers. and I would just implore people that, , that's, that's what we really need.
[00:39:12] Like actually it works. the framework works, the tool works well, and I want you to use it to install your libraries primarily because your library installation will be better, but also because that's what like that gravity is what's going to make this all way more powerful. The more packages you can install this way.
[00:39:30] That means users right now. If you have the Igniter dependency in your project, you get mixIgniter. install any hex dependency that you want. And if it doesn't have an installer, that's fine, but that's still a better experience than going and looking it up on hex and putting it in your mix. exe file, right? We need to get that as muscle memory for people to get used to that as the way that Elixir projects that you actually add, you know, dependencies and things like that. and that will be that much more likely to happen if you have an Igniter installer for your package.
[00:39:59] So that's what I want. I want people who are Working on libraries to write, even if it's a little thing, even if it's just one generates one file, it's still better. It's better than a step in your getting started guide that says, go do these things, right? And I see it everywhere. Now that I am so tuned to it, whenever I go to a package, I'm like, damn, this gives me so much easier.
[00:40:19] and we did this, all of our getting started guides have tabs at the top. Let's say with Igniter and it's a one line command and then without Igniter and it's three scrolls of stuff that you have to do to change your project. So now I'm just so keyed to it and I'm like, this is actually pretty easy to write these things as installers.
[00:40:38] And let's just let's get to that reality. Cause that's going to be attractive. That's going to drive Elixir adoption, right? if they see that Oh, I can just we want our brew install, but for Elixir, right? That's what I'm trying to get at is this one stop shop for installing packages that is a higher level of DX.
[00:40:54] So yeah, library maintainers get, get on it. 
[00:40:57] Charles: this is pretty exciting. I'm curious what you see as kind of the next steps for moving this forward. 
[00:41:03] Zach: yeah, so we have a couple like small ergonomics things. but then the next steps. For me, are getting the Ash authentication ones done. The next Igniter related project is actually, I'm going to make mix Phoenix new without all the stuff. And I'm going to show like what it looks like to do if you just add like a dash with Ecto or something like that.
[00:41:24] Right? I'm going to make those and then I'm going to pitch The relevant parties on Hey, this is what needs to happen. and I'm going to show the developer experience difference. and I'm just going to really cross my fingers that I know, I spoke with Chris about this and he's interested, he sees that is a nice UX and I also don't think that he like is in love with the way that the generators work right now.
[00:41:46] It's important that they do, but that was, You know, based on our conversation, it's like he's open to these ideas, but he's got a massive user base. It has to be a good experience for users. It has to not introduce friction. It can't be changing so often that like the mix Phoenix new can never break, right?
[00:42:00] That has to work well for everybody all the time. So there's those things to consider. So it might be a long road. But I, and my worst case scenario is that there's going to be a mix Igniter. phoenix. new, and I'll just keep my own version. I don't want to do that. and I may not just to not, I don't know.
[00:42:18] We don't want to fragment the community in any way, really. It's probably not worth it, but I do think it's such a better experience. So that's my next step is to start pitching these bigger libraries. Cause the moment it's in Phoenix, it's game over. that's, everybody's going to use that for all of the, right.
[00:42:32] All of their generators. that's the turning point. 
[00:42:35] Owen: I think that answers a question I had is, so let's say I'm, you know, I am a maintainer of WebAuthnComponents.I do have mix whack. install, which is just fun to say. so if I basically kind of refactor, rewrite that to use Igniter, the mix task name would still continue to be the same. It wouldn't be, it wouldn't be like Igniter. install.
[00:43:03] Zach: Yeah, no, it 
[00:43:04] would be exactly,
[00:43:05] Owen: yeah, so I think I actually, I see the difference now. so WebAuthnComponents would still have the same mix task name. It would function differently under the hood. You know, the output would be the same. But whenever you're in a Phoenix project and you haven't installed that yet, you would do mix Igniter. install WebAuthnComponents or WAC and then it would add that to the mix steps, then also run the Igniter mix task, that I've written inside the library itself. . And then start the server and you're good.
[00:43:36] Zach: exactly. There's one, one thing , that's worth talking about. and this is something that Chris was rightly concerned about when we first talked about it. But there are like, Igniter depends on a couple other packages, right? It depends on Rewrite, it depends on Sourcerer. And some tiny little utility libraries that those things use.I've really trimmed it down, but what, in some cases, you have to like adopt those dependencies in your project, or you have to opt into a slightly weird, one small UX experience for your end users, which is, we have a, A mechanism by which you say Igniter is an optional dependency of your project. And then you can, you write your mix tasks in such a way that if Igniter is there, then they just do what they're supposed to do.
[00:44:23] And if Igniter is not there, it just shows an error and says, please first install Igniter, and then you can use these, packages. And what that enables is that the end user can add Igniter as a dev dependency. But if they just use your package. Your package can't add Igniter statically as a dev dependency.
[00:44:44] So that means it ships in their production code, which honestly, like for Ash, I just, I don't care. I just put it, it's like three small libraries. That don't run applications, they don't make any changes, and they all depend on mix, which isn't even there in prod in the first place. So you're just, the code would just fail if you tried to run it, you know.
[00:45:03] personally, I thought it was worth that, but that may be the case that for other library authors who are more conservative about dependencies, they may need to, they may have to make that decision. If you want it to be an optional dependency for your users and let them use it as a dev dependency, then you have to do it that way.
[00:45:18] Fun fact, though. We have a mix task in Igniter called mixIgniter. gen. task that generates an Igniter task, right? So that you can get started writing your installer really easily. You would add Igniter and you would say mixIgniter. gen. task whack. install, right? and you can pass a flag that says dash dash optional, and it will do the, the logic I just said, it will generate a task that fails if Igniter's not there and pass, and so that sort of stuff is all, The, again, the DX of making that choice is fine for you, but, that is the one sort of addendum to like, oh, it all just works magically.
[00:45:54] Charles: So then do you have any regrets from building this or things you learned along the way? 
[00:46:00] Owen: oopsies? 
[00:46:01] Zach: I don't know. I, it's not really, it's been, that's again, the kind of, the reason I like it is it just keeps like working. After that initial assumption where I didn't, and then having to fight mix to install dependencies and stuff, it keeps growing nicely. Like, as I add new abstractions, they keep, we have a whole, like I mentioned, we have things like, you can say Igniter.
[00:46:23] libs. phoenix. addscope. And so, like, mount some routes in the user's router. and they're building on, less complicated things. Or on lower level, tools, like you can say, go to a module that uses this, and then you can say, go to a function call in the same scope that is this.
[00:46:39] So we can go find like the first scope in a Phoenix file and put it above that. And then we, and you can say argument equals, if you're at a function in your, so it's like this, all these things, like they keep building up to actually just really easy to use tools. So this might be, it's kind of like the exciting thing is this might be one of the only projects I've had that.
[00:46:57] Just unambiguously is going well. Where like things, everything just keeps working and I keep not, and yeah, I don't find flaws or you know, there's flaws, there's bugs, that sort of thing. Right. But the pattern is just growing so nicely.
[00:47:11] Charles: It's great.
[00:47:13] Owen: Yeah, I think when I started working on generators, so just to back up a little bit, I remember being aware of the ability to copy in the default Phoenix templates. You'd have to go to the Phoenix repo, dig around, find them. Cause they're, Kinda in a weird spot, if you are a person who uses the, the Phoenix generators like Phoenix Gen, context or Phoenix Gen Live and that kind of stuff, or you wanted to enable that for a team, say, but you want it to like override the defaults, there's a way to do that.
[00:47:45] And it's a little bit tedious. It's not super documented. but one thing to know there is whether you're taking those templates and modifying them, or you're just generating a generator, you're writing a generator for the first time, these are EEX templates. At the end of the day. So if one thing that Igniter is doing is it's actually taking the AST and modifying it, but if you have like a file, you have some code that you want to just inject, like the module name and put that into a folder somewhere, that's essentially what the Phoenix templates are doing.
[00:48:18] Those are just EEX templates. You pass in a signs, just like you pass in a signs to a controller or to a LiveView. If any of our listeners are a little bit intimidated by what we're talking about, that's, I think, I'll just point that out as a starting point. Like you don't necessarily need to understand the complexity of the AST itself.
[00:48:38] If you start by just like writing a very simple template file and then Learning how to use, you know, MixGenerator. CopyTemplate, and how that will, like, if you pass in the signs, how it looks just like an HTML template, almost like that will help you get up to speed, I think, really quickly. And then you're kind of off to the races.
[00:48:57] Then you start to dig through the, module, which is a fun journey and macros and ASTs. And then you start to look at things like Sourcerer and maybe Igniter if you're going really deep on this. 
[00:49:10] Zach: yeah, it is worth pointing out, like a lot of the stuff I'm talking about is that sort of advanced mode, but we actually, we also have Igniter. copytemplate. So you, you can have an EEX template and it will, the only difference is you just get a slightly different experience. It doesn't do it right then.
[00:49:24] It like stages the change to show the user a diff, right? so yeah, that, and that is kind of the most basic version of getting started is just saying, you Copy this template into this file location. And that's like the 80 percent that was always easy and is easy now too, with slightly better DX.
[00:49:42] It's that 20 percent that was impossible before that's now reasonably easy now. That's like where, what we've sort of added, but a lot of people don't ever have to touch that stuff. 
[00:49:52] Owen: Nice. I mean, we could go on forever talking about code generation and the ways that we're trying to simplify the developer experience for ourselves, which I think is always a great thing, and also for the people we work with, the people in our community. Deeply appreciate all the work you're doing here.
[00:50:10] Also just a quick shout out to you know, your blog article is super well written. I love to see articles that like are, you know, like we're talking about code. Yes. But they're also plain English. Like I feel like I could hand it off to developers of different experience and they would actually be able to digest and see the value of what you're saying.
[00:50:28] And it's not too terse, but it's not too verbose. So like, Very well balanced. 
[00:50:34] Zach: Thank you.
[00:50:35] Owen: I guess the last thing I'll say here is I'm looking at my source code. I see an unless that is going to get ripped out as soon as I run that format command here in a few weeks, maybe in a few months. just, note to self, I got some work to do there.
[00:50:48] before we wrap up here, do you have any final plugs or, call outs for the audience here? 
[00:50:53] Zach: I do have one small thing to say is just for fun. I'm going to write a mixIgniter. removeunless so I can do it before the language comes out. but, but yeah, no, as for my actual call outs, I do, I've two things I'd like to mention. So we have, a book we're working on it. , we're making very good progress.
[00:51:10] We've been talking about it forever. I know everybody's like, when's the book going to come out?but, building web applications with Ash framework. It's going really well, we're actually getting people Other than us are going to have some, have their hands on the book at some point, like pretty soon, which is really exciting.
[00:51:26] and I just want to shout out Rebecca who is doing an amazing job. Rebecca is my coauthor and she's the one doing most of the work on the book, just so everybody understands. So I feel weird saying she's my coauthor. I usually just fix the bugs that she finds. So I just want to say shout out to Rebecca doing a wonderful job, an amazing writer, and really, I think this is going to be far and above the, you know, I'm the most authoritative source on learning Ash and learning all the various pieces and the packages and the tools that we provide to build web applications.
[00:51:53] and I'd also like to shout out my company, Alembic. Alembic funds my work on Ash and my attendance at conferences. Alembic is, an international software agency with deep expertise in web development, distributed systems, and wrangling complex software requirements. So we combine great people with leading tech to help businesses build ambitious software applications simply, efficiently, and sustainably.
[00:52:15] And if you couldn't tell, I was reading that off of a note that I had written to myself earlier. but yeah, if you need any help, come find us. we've got really good people. We can help. 
[00:52:58] Owen: awesome. Well, you have ignited a very fun conversation, so thank you for that. Congrats, Charles. You got a great pun in there. I forget what it was. But we've got to, got to shut it down, so we're going to put out the fire. Thank you, Zach, for joining us. And to the audience, we will be back next week with more Elixir Wizards. Hey, this is Yair Flicker, president of SmartLogic, the company that brings you this podcast. SmartLogic is a consulting company that helps our clients accelerate the pace of their product development. We build custom software applications for our clients, typically using Phoenix and Elixir, Rails, React, and Flutter for mobile app development. 
[00:54:09] Outro: Elixir Wizards is a production of SmartLogic. You can find us online at smartlogic.io, and we're @smartlogic on Twitter. Don't forget to like, subscribe, and leave a review. This episode was produced and edited by Paloma Pechenik for SmartLogic. Thanks for joining us in the Creator's Lab, and we'll see you next week for more stories of innovation with Elixir.
[00:54:29] Yair: We're always happy to get acquainted even if there isn't an immediate need or opportunity. And, of course, referrals are always greatly appreciated. Please email contact@smartlogic.io to chat. Thanks, and have a great day.