• 10 dec 2017: forum version update. In case of issues use this topic.
  • 30 nov 2017: pilight moved servers. In case of issues use this topic.
Hello There, Guest! Login Register


Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
pilight v8.0 - Current work in progress
#1
I though it's maybe a good idea to tell you all what is happening with the project since not much activity can be seen on github currently.

The whole pilight internals are being rewritten to incoorporate the current features and improved functionality.

Internal event driven actions
Since the beginning of pilight, all events were communicated through specific function calls that were too much intertwined. For example, whenever the configuration was updated, a function was called evaluating all events, and a function was called to update the webserver. So for every new code that relied on config updates, a new function call had to be added in the pilight core.

The new event driven approach overcomes this by implementing a producer/consumer pattern. So the config update functionality now triggers the REASON_CONFIG_UPDATE event. On the other side various parts of pilight react on this event. So the core doesn't have to be changed when new functionality is released.

Threadpool
Since the beginning of pilight, all parallel code was run in a separate thread. This could lead to over 20 threads in complex setups. The new pilight code uses a threadpool. Functions are added to the threadpool awaiting execution and a fixed number of thread (workers) will execute them in parallel. pilight will dynamically add new workers when needed and remove them when they are idle for too long. There will never be more workers than there are number of cores.

Centralized socket handling
Since the beginning of pilight, all sockets communication was implemented on its own. Because, pilight uses non-blocking sockets, each implementation also used its own select statement to wait for events. The new approach uses a centralized socket handling library that processes all sockets at once. So all select statements were reduced to just one which grealy reduces resources. Socket events are also communicated in this library as events e.g. EV_SOCKET_SUCCES, EV_CONNECT_SOCKET, EV_READ, EV_WRITE.

Custom webserver implementation
The latest webserver implementation used by pilight (Mongoose) had some big limitations:
- SSL could not be easily ran on PolarSSL but only on OpenSSL.
- Adapting Mongoose to use a centralized socket pool was quite impossible.
- Adapting Mongoose to use the threadpool was quite impossible.
- Mongoose had a big overhead in functionality already present in pilight.

I thereby decided to drop Mongoose and created my own webserver (by reusing some of Mongoose and others code). This new webserver perfectly fits the new pilight codebase and also implements SSL by default.

Storage independent configuration
The pilight configuration has been rewritten to fix the new pilight core and has become storage independent. This means that the storage backened is now a module. The current codebase still only implements JSON as the storage backened, but this can easily be any other file based a database storage. This also allows for longitudinal data storage. The new config is also ready to implement on-the-fly config editing (through an enhanced API).

Respect UUID for all types of devices
In previous versions you could assign a UUID to a device so only a specific node would react on changes. This UUID setting only worked for relays and sensors. In the new version, the UUID setting will be respected for all devices. This means also with sending 433.92Mhz codes. This prevents interferens between nodes when sending 433.92Mhz.

Multiple AdHoc networks
In old versions you could only have one pilight adhoc network with auto-discovery. All other networks should be interconnected by using the IPTongueort settings. In the new version, multiple ad-hoc networks can be created with multiple pilight master daemons. This is done by naming each pilight daemon.
Code:
"name": "NAS"
Then you define that a specific daemon is a server or a client:
Code:
"adhoc-mode": "server"
or
Code:
"adhoc-mode": "client"

Then on the client you specify to which daemon the client should connect:
Code:
"adhoc-master": "NAS"

A client will only connect to the specific named server. Auto-reconnect will still work as in previous versions.

Using SSDP to control multiple pilight instances
In previous versions, only one pilight daemon could have its SSDP server enabled so it is auto-discoverable. In the new pilight version all master daemons will have its SSDP server enabled. This allows us to control each daemon more easily:
Code:
~ # ./pilight-control -d switch -s on
[ #]          server:port  name
[ 1]      10.0.0.141:5000  pi
[ 2]      10.0.0.140:57772 NAS
To which server do you want to connect?:

You can also directly control a specific named pilight instance by using the instance argument -I:
Code:
~ # ./pilight-control -d switch -s on -I NAS

Current status
The problem of these new functionality is that it affects everything in pilight. This also means that almost everything in pilight has to be rewritten. So all sockets, all threads, and all config related code. That's also the reason i can't commit intermediate code, because it all depends on each other.

Different visions on programming
So what i'm constantly trying to prevent is code like you can see here inside Domoticz (with all due respect, because i used their codebase as a reference for my ZWave implementation): https://github.com/domoticz/domoticz/blo...nZWave.cpp

Inside this Domoticz ZWave code, all storage calls, device types (e.g. switches, dimmers, thermostats), webserver calls, core calls are intertwined into one big spaghetti. Domoticz is a great piece of software, but in my vision of programming, the radical modulair approach is the holy grail, in contrast to what you see in Domoticz. That's also why pilight takes it (massively) slow.

What's Done
- Threadpool
- Eventpool
- Timerpool
- Webserver with SSL
- Storage independent config
- Centralized non-blocking IO.
- Protocols
- Ad-hoc Network
- Actions

Work in progress
Most work is done, currently busy testing everything locally and squashing bugs

Roadmap
I can't give any. I trying to combine a great new job and moving with the massive rewrite. But as you can see, i'm currently working on the details. Conclusion, pilight is alive and well, just not so online Smile
 
Reply
#2
Sounds pilight is getting better and better! Thanks for all the works on this!

One single question at this point of time/details, from previous pilight version changes I learned pilight was also changed at some API level. Can you estimate about the upcoming changes for v8.0 if there are API changes required?

Guenter - Author of piSchedule ... see here
 
Reply
#3
API probably won't be affected.
 
Reply
#4
Let's continue with the same question here: http://forum.pilight.org/Thread-Openwrt-...6#pid15536
 
Reply
#5
First results performance wise. The CPU/RAM resources used with the same config between pilight versions.

Version 7
Code:
cpu: 2.247484%, ram: 1.598692%
After rewrite
Code:
cpu: 0.874782%, ram: 2.752595%

The CPU usage has been reduced about 2.5 times, the RAM usage has increased 1.7 times. I'm wondering what this numbers will be in more complex setups like those of @terrar. I expect the CPU and RAM to rise faster in version 7 than after the rewrite.

Also, the CPU usage is the most important figure because that reduces the overall performance of the system. Unused RAM is merely a waste of resources (but then again shouldn't be wasted either).
 
Reply
#6
How to test 8.0?
Any url?
 
Reply
#7
It's not online yet. Currently running some durability tests.
 
Reply
#8
My test-rpi will be reactivated as soon as the first release of pilight8 is seen ;-)
Terrarium:  RPi Model B Rev 2 / pilight 8.1.2 / stretch
Aquarium: RPi Model B Plus Rev 1.2 / pilight 8.0.6 / jessie
 
Reply
#9
Will there be changes in the API? I know it has been answered long time ago but there might have been a change in plans that I'm not aware of.

Verstuurd vanaf mijn LG-D855 met Tapatalk
 
Reply
#10
No, maybe some additions but no changes.
 
Reply
  


Possibly Related Threads...
Thread Author Replies Views Last Post
  Real 433Mhz Remote is disturbed by pilight service henne111 2 147 10-06-2019, 06:18 PM
Last Post: wo_rasp
Tongue (solved) pilight 8.1.5-1-gc0a175e0 Chrashes fleisch 5 568 10-03-2019, 01:15 PM
Last Post: fleisch
  Starting pilight on boot: "cannot bind to the SSDP multicast network" pilino1234 4 336 09-29-2019, 02:08 PM
Last Post: tomk
  pilight for Raspbian Buster (raspberry pi 4) ? starob 29 2,419 07-15-2019, 08:45 PM
Last Post: curlymo
  pilight-receive Filteroption not working Alex 2 562 07-14-2019, 08:35 AM
Last Post: Alex
  pilight usb nano format conversion ettman8 2 449 07-14-2019, 08:32 AM
Last Post: curlymo
  pilight 8.1.4 crashes after some hours Ulrich.Arnold 47 2,670 06-29-2019, 08:58 PM
Last Post: curlymo
  Raspberry PI, gpio-ir-tx and pilight not starting lordslash 5 853 06-11-2019, 05:19 PM
Last Post: curlymo
  pilight fails starting on boot Alex 5 747 06-09-2019, 06:02 PM
Last Post: curlymo
  Google Assistant coupled to pilight hansrijn2 4 1,235 05-29-2019, 06:54 PM
Last Post: curlymo

Forum Jump:


Browsing: 1 Guest(s)