sending large file using a second task

[joshuaberry]

Hi,
I am quite new to writing for TCP/http/web etc so I might be quite off in my expectations. I am trying to send a large file to the browser using a separate task so that I can free up the HTTP task to resume serving pages, and not be held up while the large file is being sent. I am passing the socket that the download request is made on to the second task and writing the data to it, however as soon as the http_gethandler function (MyDoGet) returns (after starting the secondary task) the web browser stops the download due to the socket being released (I think..?).
I found this forum post (http://forum.embeddedethernet.com/viewt ... f=5&t=1650) and it seems that returning '2' prevents the socket from being released for a few seconds but the http task is still being blocked, the download is still cancelled..

Has anyone done something like this, or have any tips? I can paste code if required.

Thanks in advance,
Josh

[dciliske]

If you can put up some code I'll take a look at it and try and dive into what's triggering the failure. If you're returning a 2 it should free up the HTTP task. However, you'll want to make sure that your secondary task has a lower priority/higher number than the HTTP task to prevent it from blocking it. Also, there really isn't any form of internal rate limiting on network communications, so that transfer is going to chew up all the bandwidth it can, unless your software implements it's own limiting. This could be what's happening. I'd need to see a packet dump of the communications / run the app and tinker with the source to be able to say more.

[joshuaberry]

Thanks Dan. I'm pretty much using the custom GetHandler and SendFragment functions from the examples. I've got the secondary task running with priority 52 (http task = 45), and I just tried inserting a OSTimeDly(1) in the loop to free it up a bit but that didn't change anything. It seems as though even when MyDoGet returns 2, the socket that is being used in the secondary task is released as I can see it being used on other http requests. Is there some way of keeping the socket open/alive?

Here is most of the Get handler and the secondary task function. I've stripped out the NB example code that I haven't changed, although if you need more I can put more up.

Thanks

Code: Select all

int fileDownloadSocket;
F_FILE *f fileToDownload;
int MyDoGet( int sock, PSTR url, PSTR rxBuffer )
{
	  // 'MyDoGet' code from NB example...
	  
   if ( f_chdir( dir_buffer ) == F_NO_ERROR )
   {
      if ( name_buffer[0] == 0 )
      {

	  // 'MyDoGet' code from NB example...
	  
      }
      else  // A file name was specified in the URL, so attempt to open it
      {

		 //iprintf("  Attempting to open file \"%s\"...", pName);

		 fileToDownload = f_open( pName, "r" );

		 if ( fileToDownload != NULL )
		 {
			len = f_filelength( pName );


			fileDownloadSocket = sock;

			Byte result = OSTaskCreate( SendFragment_TaskFunction,
				   ( void * ) &fileDownloadSocket,
				   ( void * ) &FileSendingTaskStack[USER_TASK_STK_SIZE],
				   ( void * ) FileSendingTaskStack,
				   DL25Class::HTTPFileSendingPriority ) ;	// HTTPFileSendingPriority = 52


			if(result == OS_NO_ERR)
			{

			}
			else
			{
			//TODO: File sending thread start error
			}

			//iprintf(" File sent to browser\r\n");
			
		  // From Netburner forum (http://forum.embeddedethernet.com/viewtopic.php?f=5&t=1650):
		  // "Anyways, the reason for the return value is once upon a time it was requested that the get handler have the ability to not close the socket when
		  //  it finished (like transferring a 500MB file using a secondary task). The return value determines what to do. In the event that the get handler returns 2,
		  //  the webserver will not terminate the connection, with the understanding that the socket is now owned by a different task. And no, I don't know why 2 was chosen and/or why
		  //  this wasn't documented. 0 and 1 appear to have, once upon a time, been used in internal diagnostics, but are as of now unused."

			iprintf("http get: URL: %s \nreturn value: %d\nsocket: %d\n",url, 2, sock);
			return 2;

		 }
//		 else
//		 {
//			//iprintf(" file does not exist on flash card, will look in compiled application image\r\n");
//		 }
    	  

         // If the word "DIR" is specified in the URL at any directory level,
         // this code will result in a directory listng.
         if ( httpstricmp( pName, "DIR" ) )
         {
            WebListDir( sock, dir_buffer );
            iprintf("http get: URL: %s \nreturn value: %d\nsocket: %d\n",url, 0, sock);
            return 0;
         }
      }
   }


   int retVal = ( *oldhand ) ( sock, url, rxBuffer );

   //iprintf("http get: URL: %s \nreturn value: %d\nsocket: %d\n",url, retVal, sock);
   return retVal;
}

void SendFragment_TaskFunction(void* sock)
{

	 //iprintf("  Attempting to open file \"%s\"...", pName);
	 //F_FILE *f = f_open( pName, "r" );

	 int* pSocket = (int*)sock;

	 iprintf("send fragment started on socket: %d..", *pSocket);
	 if ( fileToDownload != NULL )
	 {
		//long len = f_filelength( pName );

		SendFragment( *pSocket, fileToDownload, len, false );

		f_close( fileToDownload );
	 }
	 else
	 {
		 iprintf("File open failed..\n");

	 }

	 iprintf("send fragment thread finished  Socket: %d\n", *pSocket);

}

[dciliske]

Good news: Found the problem. Bad news: it's not yours (per se).

The issue is that we don't release the socket from the webserver properly when we short circuit out of the cleanup code. You'll need to add the line

Code: Select all

ActiveHTTPSocket = 0;

inside the block after the check of the HTTPRequest.Response return code on line 282 of 'http.cpp'.

The block with modification will look like:

Code: Select all

                  if ( HTTPRequest.Respond( ActiveHTTPSocket ) == 2 )
                  {
                     ActiveHTTPSocket = 0;
                     continue;
                  }

Also, make sure that your task will actually run. Using the default EFFS-HTTP example, your task priority gets overrun by UserMain().

-Dan

[joshuaberry]

Yes! That worked exactly how I want - file downloading with some AJAX updates - thank you very much. I've changed the main() priority to MAIN_PRIO, so I think thats ok. It's all running how I expected it to anyhow.

This is my first time posting to this forum, thanks for such a quick response.

Josh