• 10 dec 2017: forum version update. In case of issues use this topic.
  • 30 nov 2017: pilight moved servers. In case of issues use this topic.
Hello There, Guest! Login Register


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
http library doesn't properly handle big response
#11
Great!

I will try to create the lua http action asap.

Did you already think about adding a logging possibility for the actions (and functions)?

For actions like write it is useful to be able to report run time issues, like a file or folder that can no longer be opened.
And I am also often using logging to help with debugging.
 
Reply
#12
I looked around in the lua http library and the existing actions working with devices.

I came to the conclusion that, to be able to create an action working with a device type that doesn't exist yet (generic_http in my case), besides the protocol itself, also a a lua device must be created in pilight/libs/pilight/lua/devices, providing methods to set and get things like code, size, mimetype and data.

Is that a correct conclusion?

I decided to create the http action first without a device (more or  less how sendmail is working), so I can test and debug the interaction with the lua http library. Once that is working I then can dive into the creation of a lua device for it.

When trying to "steal" some ideas from the sendmail lua action, I noticed that in the callback error() is being called when sending was successful. I guess that this is a bug.
 
Reply
#13
The error function is currently used for logging. When the pilight interface for logging is done i will change that.
 
Reply
#14
OK, then I will use error() for logging too.

In the meantime I have created the http action (without really handling the callback yet).

I tested http GET and the request as such seems to work, but then a segfault occurs in  plua_network_http_callback. I have seen diffrerent segfaults, depending on the request (existing url, non-existing url), but they were always in  plua_network_http_callback.

I will do some more testing, so I can give you more detail.
 
Reply
#15
Some  issues I have found with the lua http library:

If an url is called that doesn't exist the segfault below occurs. It seems logical that this happens, because when the url cannot be found, there  will be no content and STRDUP will give a segfault.

Code:
[Jul 03 08:49:35:491053] pilight-daemon: NOTICE: getaddrinfo: www.someurl.nl, Name or service not known
[Jul 03 08:49:35:491175] pilight-daemon: ERROR: host2ip
[Jul 03 08:49:35:491219] pilight-daemon: DEBUG: lua http on state #1

Thread 9 "pilight-daemon" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x720ff470 (LWP 11375)]
strlen () at ../sysdeps/arm/armv6/strlen.S:26
26      ../sysdeps/arm/armv6/strlen.S: Bestand of map bestaat niet.
(gdb) macktrace
Undefined command: "macktrace".  Try "help".
(gdb) backtrace
#0  strlen () at ../sysdeps/arm/armv6/strlen.S:26
#1  0x76408928 in __GI___strdup (s=0x0) at strdup.c:41
#2  0x768327ec in plua_network_http_callback (code=404, content=0x0, size=0, type=0x0, userdata=0x49a38) at /home/pi/pilight/libs/pilight/lua/network/http.c:309
#3  0x767d2acc in http_process (type=0, url=0x49850 "http://www.someurl.nl/", conttype=0x0, post=0x0, callback=0x76832524 <plua_network_http_callback>, userdata=0x49a38)
   at /home/pi/pilight/libs/pilight/core/http.c:856
#4  0x767d2be8 in http_get_content (url=0x49850 "http://www.someurl.nl/", callback=0x76832524 <plua_network_http_callback>, userdata=0x49a38)
   at /home/pi/pilight/libs/pilight/core/http.c:873
#5  0x76832f0c in plua_network_http_get (L=0x763151c0) at /home/pi/pilight/libs/pilight/lua/network/http.c:432
#6  0x76561708 in ?? () from /usr/lib/arm-linux-gnueabihf/libluajit-5.1.so.2
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) frame 2
#2  0x768327ec in plua_network_http_callback (code=404, content=0x0, size=0, type=0x0, userdata=0x49a38) at /home/pi/pilight/libs/pilight/lua/network/http.c:309
309                     if((data->data = STRDUP((char *)content)) == NULL) {
(gdb)

If an url is called that does exist different errors occur. However those errors ar not consistent. This one I have seen only once till now:

Code:
[Jul 03 09:32:46:358158] pilight-daemon: DEBUG: rule #3 httptest was parsed in 0.000073 seconds
[Jul 03 09:32:46:576231] pilight-daemon: DEBUG: lua http on state #0
PANIC: unprotected error in call to Lua API (cannot find  lua module)
[Thread 0x702da470 (LWP 11773) exited]
[Thread 0x70ada470 (LWP 11772) exited]
[Thread 0x712da470 (LWP 11771) exited]
[Thread 0x71ada470 (LWP 11770) exited]
[Thread 0x722ff470 (LWP 11769) exited]
[Thread 0x72cff470 (LWP 11768) exited]
[Thread 0x734ff470 (LWP 11767) exited]
[Thread 0x73eff470 (LWP 11766) exited]
[Thread 0x746ff470 (LWP 11765) exited]
[Thread 0x74eff470 (LWP 11764) exited]
[Thread 0x75a74470 (LWP 11763) exited]
[Thread 0x76274470 (LWP 11762) exited]
[Inferior 1 (process 11757) exited with code 01]
(gdb) backtrace
No stack.
(gdb)

but with the same request, this one happens almost every time.

Code:
[Jul 03 09:34:41:994777] pilight-daemon: DEBUG: rule #3 httptest was parsed in 0.000072 seconds
[ Jul 03 09:34:42:31184] pilight-daemon: DEBUG: lua http on state #0

Thread 1 "pilight-daemon" received signal SIGSEGV, Segmentation fault.
0x768325b0 in plua_network_http_callback (code=200,
   content=0x4aaa0 "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"nl\"><head><meta content=\"text/html; charset=UTF-8\" http-equiv=\"Content-Type\"><meta content=\"/images/branding/googleg/1x/goo"..., size=11263, type=0x49c98 "text/html", userdata=0x49210) at /home/pi/pilight/libs/pilight/lua/network/http.c:268
268             switch(state->module->type) {
(gdb) backtrace
#0  0x768325b0 in plua_network_http_callback (code=200,
   content=0x4aaa0 "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"nl\"><head><meta content=\"text/html; charset=UTF-8\" http-equiv=\"Content-Type\"><meta content=\"/images/branding/googleg/1x/goo"..., size=11263, type=0x49c98 "text/html", userdata=0x49210) at /home/pi/pilight/libs/pilight/lua/network/http.c:268
#1  0x767d1530 in http_client_close (req=0x49bf0) at /home/pi/pilight/libs/pilight/core/http.c:475
#2  0x767d16ac in poll_close_cb (req=0x49bf0) at /home/pi/pilight/libs/pilight/core/http.c:522
#3  0x767c93bc in eventpool_update_poll (req=0x49bf0) at /home/pi/pilight/libs/pilight/core/eventpool.c:420
#4  0x767ca210 in uv_custom_poll_cb (req=0x49bf0, status=0, events=1) at /home/pi/pilight/libs/pilight/core/eventpool.c:757
#5  0x767a843c in uv__poll_io (loop=0x76fa7f50 <default_loop_struct>, w=0x49c24, events=1) at /home/pi/pilight/libs/libuv/unix/poll.c:64
#6  0x767a1cdc in uv__io_poll (loop=0x76fa7f50 <default_loop_struct>, timeout=961) at /home/pi/pilight/libs/libuv/unix/linux-core.c:400
#7  0x76798390 in uv_run (loop=0x76fa7f50 <default_loop_struct>, mode=UV_RUN_DEFAULT) at /home/pi/pilight/libs/libuv/unix/core.c:362
#8  0x0001cd34 in main (argc=2, argv=0x7efff7a4) at /home/pi/pilight/daemon.c:3506
(gdb) frame 0
#0  0x768325b0 in plua_network_http_callback (code=200,
   content=0x4aaa0 "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"nl\"><head><meta content=\"text/html; charset=UTF-8\" http-equiv=\"Content-Type\"><meta content=\"/images/branding/googleg/1x/goo"..., size=11263, type=0x49c98 "text/html", userdata=0x49210) at /home/pi/pilight/libs/pilight/lua/network/http.c:268
268             switch(state->module->type) {
(gdb)

I tried to get some more info on what is happening by adding a debug line in /home/pi/pilight/libs/pilight/lua/network/http, but that causes to change the statement where the segfault occurs. This makes me think of a timing issue.

Code:
[Jul 03 09:41:56:505394] pilight-daemon: DEBUG: rule #3 httptest was parsed in 0.000077 seconds
[Jul 03 09:41:56:541001] pilight-daemon: DEBUG: lua http on state #0
[Jul 03 09:41:56:541059] pilight-daemon: DEBUG: state->module->name = http, state->module->type = 3

Thread 1 "pilight-daemon" received signal SIGSEGV, Segmentation fault.
strlen () at ../sysdeps/arm/armv6/strlen.S:26
26      ../sysdeps/arm/armv6/strlen.S: Bestand of map bestaat niet.
(gdb) backtrace
#0  strlen () at ../sysdeps/arm/armv6/strlen.S:26
#1  0x765a2ba8 in lua_getfield () from /usr/lib/arm-linux-gnueabihf/libluajit-5.1.so.2
#2  0x768326d8 in plua_network_http_callback (code=200,
   content=0x4aa50 "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"nl\"><head><meta content=\"text/html; charset=UTF-8\" http-equiv=\"Content-Type\"><meta content=\"/images/branding/googleg/1x/goo"..., size=11273, type=0x4a260 "text/html", userdata=0x49e08) at /home/pi/pilight/libs/pilight/lua/network/http.c:285
#3  0x767d1530 in http_client_close (req=0x4a380) at /home/pi/pilight/libs/pilight/core/http.c:475
#4  0x767d16ac in poll_close_cb (req=0x4a380) at /home/pi/pilight/libs/pilight/core/http.c:522
#5  0x767c93bc in eventpool_update_poll (req=0x4a380) at /home/pi/pilight/libs/pilight/core/eventpool.c:420
#6  0x767ca210 in uv_custom_poll_cb (req=0x4a380, status=0, events=1) at /home/pi/pilight/libs/pilight/core/eventpool.c:757
#7  0x767a843c in uv__poll_io (loop=0x76fa7f50 <default_loop_struct>, w=0x4a3b4, events=1) at /home/pi/pilight/libs/libuv/unix/poll.c:64
#8  0x767a1cdc in uv__io_poll (loop=0x76fa7f50 <default_loop_struct>, timeout=962) at /home/pi/pilight/libs/libuv/unix/linux-core.c:400
#9  0x76798390 in uv_run (loop=0x76fa7f50 <default_loop_struct>, mode=UV_RUN_DEFAULT) at /home/pi/pilight/libs/libuv/unix/core.c:362
#10 0x0001cd34 in main (argc=2, argv=0x7efff7a4) at /home/pi/pilight/daemon.c:3506
(gdb) frame 2
#2  0x768326d8 in plua_network_http_callback (code=200,
   content=0x4aa50 "<!doctype html><html itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"nl\"><head><meta content=\"text/html; charset=UTF-8\" http-equiv=\"Content-Type\"><meta content=\"/images/branding/googleg/1x/goo"..., size=11273, type=0x4a260 "text/html", userdata=0x49e08) at /home/pi/pilight/libs/pilight/lua/network/http.c:285
285             lua_getfield(state->L, -1, data->callback);
(gdb)
 
Reply
#16
You're not using the latest staging code. Can you do and recheck. Also provide me with concrete test cases.
 
Reply
#17
I have not completely installed the latest staging code, but I think I have implemented all fixes and updates.
Anyway, I will reinstall and test again.
 
Reply
#18
Did you get the chance to test things again?
 
Reply
#19
Sorry, Yes I did, but I forgot to send the post I composed already  yesterday  Blush .


I did a fresh clone and install and left out all my own additions except the http.lua script.
The results are the same as I posted before.

So segfaults for every http get request wether or not the url called exists:

The testcases are simple:

call http://www. google.nl/ (existing url) -> segfault
call http://www.xyz.xx/ (non existing url) -> segfault

I am using my attached http.lua script (handling of the response not yet implemented) to do the requests.


Attached Files
.zip   http.zip (Size: 1.31 KB / Downloads: 2)
 
Reply
#20
Both errors should be fixed
 
Reply
  


Possibly Related Threads...
Thread Author Replies Views Last Post
  ][solved]Segfault when retrieving big chunked http message Niek 21 5,306 11-29-2018, 03:17 PM
Last Post: curlymo
  http code 301 causes segfault Niek 3 3,201 08-14-2018, 06:57 PM
Last Post: curlymo
  no callback on some http requests Niek 20 5,916 07-28-2018, 10:22 AM
Last Post: Niek

Forum Jump:


Browsing: 1 Guest(s)