Need some help understanding code flow. IE: ESPHome, YAML, and code pulled from github.
Hi!
**My ask up front: **Can someone walk me through the process of how a github project written in python winds up interacting with YAML on an ESP32? Looking for some orientation so I can "RTFM" better.
Background: I have Espectre up and running in my house, which is cool. What I also have in my house is LED crown molding controlled by WLED.
I think the ESP-32s I'm using for the lighting have enough juice to run both items of code (and probably a few other sensors). However, I don't know how to combine them.
What I'm specifically confused about is when I provision a device with ESPHome, I'm not actually clear on what it does. As in: I gather it's making a framework specific to the board to handle running the code in the YAML, but I'm not sure what that looks like specifically. Which means I don't know how to take the code from WLED or ESPectre and also include it (is it as simple as an include?) so that I can make YAMLs that utilize the functionality those to projects provide.
I am especially confused what is happening when YAML code calls specific github projects. Is it literally pulling code from the repository?
For context I have decent capability and experience, but mostly within code, bridging multiple architectures like this is new. I have stumbled on for years doing some fairly advanced projects, but I mostly am a very determined monkey banging on a type writer.
7 replies
ESPHome is by-and-large a build system to build bespoke C++ programs. The python files you're seeing describe how that yaml file maps to the C++ files.
The core idea is that end users (including people managing custom sensors) as much as possible only have to interact with the yaml file. The yaml file lists what components you want to run and their configurations. A component is in general a subdirectory of esphome/esphome and is intended to be independent of other components.
Those github references in the yaml pull in external components from the community. It's roughly equivalent to dropping another folder into the source tree (noting again that you shouldn't usually need to have the source tree at all).
Running esphome copies all used components into a separate source tree, with substitutions and variables set according to the yaml configuration (defined by the component's python files) and builds a bespoke binary for you that will run on your device.
If you want to include code that's not already an ESPHome component, you will need to make it into one. You can do this in place in the source tree or by creating an external component yourself. You will want to use an existing component as a template for this. Have a look for if ESPHome itself or a community component already exists first because it will make your life so much easier.
TL;DR the python code converts the yaml into a (pure) C++ program.
To add on to what the other replier said, these two projects you wish to combine are not written with that intent. They are the root firmware, effectively.
Modular systems like ESPHome are written with that modularity in mind, and have underlying code APIs (not like a REST endpoint) that those modules hook into and utilize.
Even if WLED and Espectre were written to be modular firmware, they would not be compatible with each other, nor ESPHome, unless expressly written for such.
As far as the "calling Github" part: Yes, sort of. It's meant for making it easier to pull modular packages others have written for ESPHome to share with others, allowing easy inclusion of code that doesn't require redownloading files yourself every time they update it for changes in ESPHome or bug fixes.
@batmaniam #esphome uses python to read a yaml file and generate C++, and then compiles the result. In general, you can’t just trivially combine two esp32 projects into a single firmware. However, esphome has a component that accepts the #wled udp broadcast protocol from software running elsewhere on your network. This may or may not end up being a simplification - there are a small number of existing “senders”, mostly designed for coordinating large lighting installations such as Christmas light displays across multiple ESP32s.
Additionally, esphome has a number of built-in lighting effects (fewer than wled, and less flexible), which may be enough for your purposes. If so, you should be able to consolidate two esp32s into one.
Then again, the positioning of the esp32 probably plays a role in its ability to detect motion using the espectre component, so there may still be tradeoffs involved. Ie the led controller may be positioned near the ceiling where it is less able to detect movement.
This is fantastic, thank you. I had YAML backwards, as something that deeper code body referenced, not what are essentially the instructions for constructing that deeper code body. Very cool and makes some of the different YAML functions make much more sense now.
That also helped me understand the github structure: the YAML will have instructions that are specifc to the code in that data base. Question though: Why isnt it as simple as using an "include" the way you would with other libraries? If thats not an easy one I'll do reading, this is crazy helpful.
Even though it's not as simple as mashing together the two projects, I think I can Frankenstein the two together from their respective code. In particular I want to dig in a lot deeper into ESPecter. It works pretty but there are some quirks. IE: If I walk between my router and my bedroom, easy to do as they're on different floors, by bedroom will register motion and turn on the underbed lighting. So the positioning that you mentioned does matter, a lot, although closer to the ceiling might honestly give better sensitivity. It is finicky but I have been really impressed so far and it's an active project. Highly recommend a spin if you're OK with something that isn't perfect.
The number of ESP-32s I have deployed keeps growing, and ESPecter works pretty well as it is, I think if I dig in enough I can maybe do some clever stuff using the logic/info from multiple. With ~12 deployed, and plans for ~10+, that's a lot work with.
@batmaniam the reason its not as easy as just including a library is that the two projects are not designed that way.
ESPHome and WLED are not libraries, they are complete projects in their own right.
Sure, you can bodge an air filter for one car into a different one, but you can’t (easily!) bodge a whole car into a different one! #StupidCarAnalogies
ESPectre is also a project in its own right, but additionally has been made available as an ESPHome external component- ie ESPHome has a feature to copy the relevant ESPectre code and ESPHome adapter/integration code into the ESPHome tree while doing its codegen/compilation step.
I guess that makes sense, if the YAML is essentially instructions on how to write the code using a specific code base, I can see things going wrong when it's two code bases that weren't made to work with each other. I may not combine them right away, but I definitely want to dig deeply into ESPectre. There just has to be some fun you can have integrating multiple signals.
Thank you for taking the time, it makes my own reading a lot more productive!