Upload Progress Issue -- do not have "uploading" state

Hello,
I am trying to use Nginx + Upload Module + Upload Progress Module, which
I follow the link
http://wiki.nginx.org/NginxHttpUploadProgressModule#Usage

I find some issue in initial config, like X-Progress-ID should be added
in upload_pass_form_field

Now I have the last problem is upload.state is not changing during file
transfer. When upload starts, the state is “Starting”, when upload is
done, state becomes “done”. But in the period, states never becomes
“uploading”. This cause the status bar is not working.

Could anyone help me to solve this issue?
Thanks.

I post my config here:

//conf

246 location ~ /api/1.0/web/upload.php {
247 # limit_req zone=box; #set request limit for box
248 root html;
249 fastcgi_pass 127.0.0.1:9000;
250 fastcgi_param SCRIPT_FILENAME
/opt/nginx/html/script$fastcgi_script_name;
251 include fastcgi_params;
252
253 }

268 location ~ /api/1.0/web/upload_internal {
269 # Pass altered request body to this location
270
271 if ($request_method = POST) {
272 upload_pass @post_upload;
273 break;
274 }
275
276 #proxy to upstream server
277 proxy_pass http://127.0.0.1;
278 proxy_redirect default;
279
280 # Store files to this directory
281 upload_store /opt/nginx/html/upload_file;
282
283 # Allow uploaded files to be read only by user
284
285 upload_store_access user:rw group:rw all:rw;
286 upload_set_form_field $upload_field_name.name
“$upload_file_name”;
287 upload_set_form_field $upload_field_name.content_type
“$upload_content_type”;
288 upload_set_form_field $upload_field_name.path
“$upload_tmp_path”;
289
290 # Inform backend about hash and size of a file
291 upload_aggregate_form_field “${upload_field_name}_md5”
$upload_file_md5;
292 upload_aggregate_form_field “${upload_field_name}_size”
$upload_file_size;
293
294 upload_pass_iform_field
“^X-Progress-ID$|^submit$|^email$”;
295
296
297 # track uploads in the ‘proxied’ zone
298 # remember connections for 30s after they finished
299 track_uploads proxied 30s;
300
301 }

303 # Pass altered request body to a backend
304 location @post_upload {
305 rewrite /api/1.0/web/upload_internal
/api/1.0/web/upload.php last;
306 }
307
308 location ^~ /progress {
309 report_uploads proxied;
310 }

//form //

314
315 Email:

316

317
318
319
320
321


322

323

324

325
Upload Progress

326

327

// javascript //

54 interval = null;
55
56 function openProgressBar() {
57 /* generate random progress-id /
58 uuid = “”;
59 for (i = 0; i < 32; i++) {
60 uuid += Math.floor(Math.random() * 16).toString(16);
61 }
62 /
patch the form-action tag to include the progress-id */
63
//document.getElementById(“upload”).action=“/upload.php?X-Progress-ID=”

  • uuid;
    64
    document.getElementById(“upload”).action=“/api/1.0/web/upload_internal/?X-Progress-ID=”
  • uuid;
    65
    66 /* call the progress-updater every 1000ms /
    67 interval = window.setInterval(
    68 function () {
    69 fetch(uuid);
    70 },
    71 100
    72 );
    73 }
    74
    75 function fetch(uuid) {
    76 req = new XMLHttpRequest();
    77 req.open(“GET”, “/progress”, 1);
    78 req.setRequestHeader(“X-Progress-ID”, uuid);
    79 req.onreadystatechange = function () {
    80 if (req.readyState == 4) {
    81 if (req.status == 200) {
    82 /
    poor-man JSON parser /
    83 var upload = eval(req.responseText);
    84
    85 document.getElementById(‘tp’).innerHTML = upload.state;
    86
    87
    88 /
    change the width if the inner progress-bar /
    89 if (upload.state == ‘done’ || upload.state == ‘uploading’) {
    90 //if (upload.state == ‘done’ || upload.state == ‘start’) {
    91 //alert(req.responseText);
    92 bar = document.getElementById(‘progressbar’);
    93 w = 400 * upload.received / upload.size;
    94 bar.style.width = w + ‘px’;
    95 }
    96 /
    we are done, stop the interval */
    97 if (upload.state == ‘done’) {
    98 //alert(“Hello”);
    99 //alert(req.responseText);
    100 window.clearTimeout(interval);
    101 }
    102 }
    103 }
    104 }
    105 req.send(null);
    106 }

BTW, I am using
Nginx 1.0.0
upload 2.2.0
uploadprogress 0.8.0

Hello Martin,
Thank you for your reply.
I tried with your suggestion on your blog, with 0.8.2
However, I have initial error from beginning.

nginx: [emerg] no “events” section in configuration

Another thing is I use fastcgi to process PHP request.
In your blog, I do not see fastcgi backend.

I also tried 0.8.2 with my previous config, no good luck either.

Could you take a look at my config. I really do not know where
“uploading” state come from. Could you give me some hint?

Thanks,
Yanxin

The latest version of upload progress module is 0.8.2 so you should
upgrade to that to begin with. If that doesn’t solve the problem then I
documented my install and progress bar here:

That’s almost exactly the code I have in production working just fine,
so maybe you can use it as a reference to see if you are missing
anything.

Cheers,
Martin Fjordvald

Posted at Nginx Forum:

I did some debug, and realize ngx_http_uploadprogress_event_handler() is
never been called.

That’s the reason I see starting and done state, but never see
“uploading” state.

I want to know which nginx configure and html part will call

ngx_http_uploadprogress_event_handler

Thanks,
Yanxin

I print out the debug

2021 2011/04/20 17:28:07 [debug] 28372#0: *6 http script var:
“/opt/nginx/html/api/1.0/web/ upload_internal/”
2022 2011/04/20 17:28:07 [debug] 28372#0: *6 http script copy: “^@”
2023 2011/04/20 17:28:07 [debug] 28372#0: *6 http script file op
0000000000000005 “/opt/nginx/ html/api/1.0/web/upload_internal/”
2024 2011/04/20 17:28:07 [debug] 28372#0: *6 http script if
2025 2011/04/20 17:28:07 [debug] 28372#0: 6 http script regex:
"(.
)/upload$"
2026 2011/04/20 17:28:07 [notice] 28372#0: 6 "(.)/upload$" does not
match “/api/1.0/web/ upload_internal/”, client: 10.31.1.100,
server: localhost, request: “POST /api/1.0/web/
upload_internal/?X-Progress-ID=db298fe5d036a8ae19d2a55b9d1d0ca9
HTTP/1.1”, host: “10.1.4. 243”, referrer:
https://10.1.4.243/api/1.0/web/upload
2027 2011/04/20 17:28:07 [debug] 28372#0: *6 test location: “/”
2028 2011/04/20 17:28:07 [debug] 28372#0: *6 test location: “progress”
2029 2011/04/20 17:28:07 [debug] 28372#0: *6 test location: “50x.html”
2030 2011/04/20 17:28:07 [debug] 28372#0: *6 test location: ~
“/api/1.0/web/upload.php”
2031 2011/04/20 17:28:07 [debug] 28372#0: *6 test location: ~
“/api/1.0/web/.*php$”
2032 2011/04/20 17:28:07 [debug] 28372#0: *6 test location: ~
“/api/1.0/web/upload_internal”
2033 2011/04/20 17:28:07 [debug] 28372#0: *6 using configuration
“/api/1.0/web/upload_internal”
2034 2011/04/20 17:28:07 [debug] 28372#0: *6 http cl:29308 max:10485760
2035 2011/04/20 17:28:07 [debug] 28372#0: *6 rewrite phase: 2
2036 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress:
get_tracking_id
2037 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress:
get_tracking_id no header found
2038 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress:
get_tracking_id no header found, args found
2039 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress:
get_tracking_id found args: X-
Progress-ID=db298fe5d036a8ae19d2a55b9d1d0ca9 HTTP/1.1^M
2040 Host
2041 2011/04/20 17:28:07 [debug] 28372#0: *6 malloc: 0000000010B7CF40:16
2042 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress:
get_tracking_id found args: db298fe5d036a8ae19d2a55b9d1d0ca9
2043 2011/04/20 17:28:07 [debug] 28372#0: *6 trackuploads id found:
db298fe5d036a8ae19d2a55b9d1d0ca9
2044 2011/04/20 17:28:07 [debug] 28372#0: *6 trackuploads hash 5351D33B
for id: db298fe5d036a8ae19d2a55b9d1d0ca9
2045 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress: find_node
db298fe5d036a8ae19d2a55b9d1d0ca9
2046 2011/04/20 17:28:07 [debug] 28372#0: *6 upload-progress: can’t find
node
2047 2011/04/20 17:28:07 [debug] 28372#0: *6 add cleanup:
0000000010B7F670
2048 2011/04/20 17:28:07 [debug] 28372#0: slab alloc: 136 slot: 5
2049 2011/04/20 17:28:07 [debug] 28372#0: slab alloc: 00002B65BDE08000
2050 2011/04/20 17:28:07 [debug] 28372#0: *6 trackuploads: 5351D33B
inserted in rbtree
2051 2011/04/20 17:28:07 [debug] 28372#0: event timer add: 7:
15000:1303345702489
2052 2011/04/20 17:28:07 [debug] 28372#0: *6 rewrite phase: 3
2053 2011/04/20 17:28:07 [debug] 28372#0: *6 http script var
2054 2011/04/20 17:28:07 [debug] 28372#0: *6 http script var: “POST”
2055 2011/04/20 17:28:07 [debug] 28372#0: *6 http script value: “POST”
2056 2011/04/20 17:28:07 [debug] 28372#0: *6 http script equal

I look at the code, in find_node(), I find out node and sentinel are
NULL.
So it does not go into while loop.

350 static ngx_http_uploadprogress_node_t *
351 find_node(ngx_str_t * id, ngx_http_uploadprogress_ctx_t * ctx,
ngx_log_t * log)
352 {
353 uint32_t hash;
354 ngx_rbtree_node_t *node, *sentinel;
355 ngx_int_t rc;
356 ngx_http_uploadprogress_node_t up;
357
358 ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, “upload-progress:
find_node %V”, id);
359
360 hash = ngx_crc32_short(id->data, id->len);
361
362 node = ctx->rbtree->root;
363 sentinel = ctx->rbtree->sentinel;
364
365 ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, “upload-progress:
node: %V sentinel: %V”, node, sentinel);
366
367 while (node != sentinel) {
368
369 ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0,
“upload-progress: hash: %V node->key: %V”, hash, node->key);
370 if (hash < node->key) {
371
372 node = node->left;
373 continue;
374 }
375
376 if (hash > node->key) {
377 node = node->right;
378 continue;
379 }
380
381 /
hash == node->key */
382
383 do {
384
385 ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0,
“upload-progress: In while”);
386 up = (ngx_http_uploadprogress_node_t *) node;
387
388 rc = ngx_memn2cmp(id->data, up->data, id->len, (size_t)
up->len);
389
390 if (rc == 0) {
391 ngx_log_debug0(NGX_LOG_DEBUG_HTTP, log, 0,
392 “upload-progress: found node”);
393 return up;
394 }

I want to know which function to initialize node and sentinel.

One possible reason is I am using track_uploads on default port 80,
however, the fastcgi in my config is using port 9000.

Could anyone tell me how to monitor port 9000 in upload progress?

Thanks,
Yanxin

On 21/04/11 03:30, Yanxin Z. wrote:

done, state becomes “done”. But in the period, states never becomes
“uploading”. This cause the status bar is not working.

Could anyone help me to solve this issue?
Thanks.

I post my config here:

I think your config is extremely complex, there are certainly ways to
make it simpler. Remember tha the upload progress module is trick, and
your config certainly breaks it.

Also, you didn’t tell us how you were testing: remember that if you test
over localhost even with large (multi gigabyte) files, the upload will
take way less time than you can see. Make sure to either throttle your
upload (for instance do the upload with curl) so that you have a chance
to actually see it.

253 }

268 location ~ /api/1.0/web/upload_internal {
269 # Pass altered request body to this location
270
271 if ($request_method = POST) {
272 upload_pass @post_upload;
273 break;

I think the problem is here. You tell nginx to break so the
track_uploads doesn’t get a chance to run.

Hello Brice,
Thank you for your help.

I changed my conf: 1) comment out break; 2) put proxy_pass in front of
upload_pass.

However, the issue is the same. Can not track upload status.
I find pioneer encountered the same issue before.

I try to figure out the part that is not working. Could you give me some
hints?

For the uploading check, chrome has its own upload status check, I use
it to track the file uploading and compare it with upload module.
Yanxin

272 location ~ /api/1.0/web/upload_internal {
273 #proxy to upstream server
274 proxy_pass https://127.0.0.1;
275
276 # Pass altered request body to this location
277
278 if ($request_method = POST) {
279 upload_pass @post_upload;
280 #break;
281 }
282
283
284 # Store files to this directory
285 upload_store /opt/nginx/html/upload_file;
286
287 # Allow uploaded files to be read only by user
288
289 upload_store_access user:rw group:rw all:rw;
290 upload_set_form_field $upload_field_name.name
“$upload_file_name”;
291 upload_set_form_field $upload_field_name.content_type
“$upload_content_type”;
292 upload_set_form_field $upload_field_name.path
“$upload_tmp_path”;
293
294 # Inform backend about hash and size of a file
295 upload_aggregate_form_field “${upload_field_name}_md5”
$upload_file_md5;
296 upload_aggregate_form_field “${upload_field_name}_size”
$upload_file_size;
297
298 upload_pass_form_field
“^X-Progress-ID$|^submit$|^email$|^.*
id$|^sha256$|^.*ip$|^.*port$|username|filename|fileurl|vsys”;
299
300
301 # track uploads in the ‘proxied’ zone
302 # remember connections for 30s after they finished
303
304 upload_cleanup 400 404 499 500-505;
305
306 track_uploads proxied 10s;
307
308 }