IP Network driver

[Martin_Head]

Hi, I was wondering if I could pick the brains of anyone who understands the inner workings of SMSQ/E.

I'm trying to get the network working in QPC2, using the the IP device drivers.

So far I've just tried to implement the basic NETI/O commands of the standard QL. Which just about seem to be working.

I can do an OPEN neto_xx on one PC and a OPEN neti_xx on another, and use PRINT# and INPUT# to transfer data. I can also SAVE neto_xx on one PC and then run a loop INPUTing on the other to get the basic listing OK.

Now the problem. If I try to LOAD, or LBYTES neti_xx, the system crashes, There may be a bug(s) still lurking in the driver causing this, However if I write a small machine code program to open a network channel, then do a FS_LOAD trap #3 call, telling it how much data there is to load, It loads the data OK. Which suggests the driver is basically working.

So how does the SMSQ/E LOAD and LBYTES commands deal with network serial communications when there is no file header to get the length of the data to read. A quick look at the source code for SMSQ/E suggests it looks for a file header, which the SAVE neto_xx command does not seem to add to the data it sends.

I'm wondering if SMSQ/E uses another part of the network drivers (the Nx device, as in LOAD n1_flp1_filename, or the NFS_ device), to handle LOAD and LBYTES. So I am at the moment trying to add these drivers to see if it helps.

Thanks in advance

Martin Head

[tofro]

Martin,
what you seem to be doing is implementing a stacked device driver (i.e. calling the OS while in an OS trap, thus in a reentrant way). SMSQ/E is not made for that, it can get into all sorts of troubles, most of them relating to supervisor stack issues.

What you normally should be able to do from your driver is calling the below device driver directly instead of trap - ing into the OS to make that call. You'd need to find the open, close and I/O entry points from the device driver linkage block, set up the registers according to their specific requirement (it's all in the docs), and directly JSR to those addresses. I have never tried this myself, because in my driver I could find another, simpler, work-around, but I think that has the highest chance of working.

Hope this helps,
Tobias

[martyn_hill]

Hi Martin

Nice to read that someone (more brave than I!) is working towards wrapping standard SMSQ file IO around the IP stack!

The TRAP 'stacking' Tobias mentions sounds spikey and is something I'm trying to figure out for a project of my own (QLNET to USB interface for SMSQ on PC - still some way off.)

On a less sophisticated note, my own testing (across a simple serial connection) - prompted by another thread on the forum at the time - showed that that 'mini' 15-byte header that QDOS requires for non directory devices is only pre-pended by some operations.

SBYTES and its brethren appear to add the mini-header, whereas a simple SAVE won't. Makes sense.

Sure you knew that already, but you didn't mention it specifically. Is that relevant to your findings?

Depending upon what those first few bytes of the raw event stream hold, the remote end could think its waiting for a very large file and hang or worse...

[Martin_Head]

what you seem to be doing is implementing a stacked device driver (i.e. calling the OS while in an OS trap, thus in a reentrant way). SMSQ/E is not made for that, it can get into all sorts of troubles, most of them relating to supervisor stack issues.

Yes, The operating system gets confused if you try to "open" a channel while in an "open" channel routine. I had to fiddle about with the channel table to get around that.

SBYTES and its brethren appear to add the mini-header, whereas a simple SAVE won't. Makes sense.

I've not tried SBYTES from the sender yet, I will try that to see if it adds a header.

What Tobias was saying about calling a system trap while in a system trap. Is something that has concerned me, When the operating system enters the first trap it's in supervisor mode, then when it enters the second trap goes into supervisor mode, which it was already in. But when the second trap exits, does it put the processor back into user mode, thus screwing up the first trap?

But the thing is, the driver is so nearly working. I can SAVE on one computer and use a loop of INPUT# on the other computer to retrieve the basic listing. I even get a end of file error if I INPUT too far. So it seems to handling calling a system trap while in a system trap.

What I think I might do, is to add some debugging code to the driver routines to print messages to the screen, so I try to see how far the LOAD and LBYTES commands get before the system locks up.

Martin Head

[tofro]

What Tobias was saying about calling a system trap while in a system trap. Is something that has concerned me, When the operating system enters the first trap it's in supervisor mode, then when it enters the second trap goes into supervisor mode, which it was already in. But when the second trap exits, does it put the processor back into user mode, thus screwing up the first trap?

On the processor level - That shouldn't be a problem. RTE from the first trap pops both the return address and the SR register - The latter will let the CPU know it's still in SV mode.
On the OS level, when I tried (for pure curiosity) what you are doing, I have seen wrong channel IDs all around the place where the OS thought it would actually handle the "outer" channel, but was asked to work on the "inner" (or vice versa). And, if I recall that right, that did not only happen in the "open" call.

With regards to the header/headerless transfer across the "NnetX" device: The original QDOS drivers would not require a header here, because the low-level packet protocol on the QLAN would tell the system exactly how many bytes are in a packet and how much there's still to come. Maybe that's what's missing in your version?

Regards,
Tobias

[martyn_hill]

With regards to the header/headerless transfer across the "NnetX" device: The original QDOS drivers would not require a header here, because the low-level packet protocol on the QLAN would tell the system exactly how many bytes are in a packet and how much there's still to come.

Whilst I can't explain the problem Martin's seeing, nor do I think that the standard low-level NET driver does anything special with regard to advising the receiver of file-size (and other data usually found in a header).

My recent scouring of the (Minerva and TK2/SMSQ) NET driver (both access and physical layers) revealed no special handling of file lengths (or, say Dataspace) that we know finds its way across the QLNET to allow LBYTES, EXEC etc to successfully load and execute files sent across it.

The only length data handled in the NET driver seems to be the packet length - either 255 for all but the last packet, or the remainder in the last packet (skipping TK2's 1024 byte FSERVE packets for this discussion).

Serial EOF handling works well enough (hence LOAD and INPUT etc work as expected), but LBYTES etc need a specified buffer size to work with.

Looking further through the S*BASIC sources, shows that LBYTES etc will invoke fs.headr before fs.load. fs.headr itself looks for that 'mini' file-header ($FF + 14 bytes).

SBYTES shows the equivalent (set header, then send bytes).

All this points back to needing that mini-header as the first 15 bytes of the stream for LBYTES etc to function.

As for why Martin's LBYTES attempt hangs (rather than rejecting the received file due to a missing $FF) remains a little mystery...

Martin - what were the first 15 bytes of your test file?

Did you get around to testing SBYTES to send (and thus get the header pre-pended)?

[Martin_Head]

Firstly, In case of any possible confusion, I am not writing this driver from scratch, I am adapting the Network driver from the SMSQ/E source code. So it should interface with the operating system correctly.

Martin - what were the first 15 bytes of your test file?

Did you get around to testing SBYTES to send (and thus get the header pre-pended)?

I played about with the driver yesterday and I found that SBYTES neto_x adds the 15 bytes file header, but SAVE neto_x does not.

I also found that while LBYTES neti_x and LOAD neti_x crashes the system. If I do

OPEN#4,neti_x
LBYTES#4 (or LOAD#4)
CLOSE#4

Works OK.

A thought that occurred to me. Is the Supervisor stack overflowing into somewhere it should not. I seem to remember reading that the Supervisor stack is only 64 bytes. Without checking, I think the I/O routine uses a good 40 odd bytes, and I do use few more bytes of the Supervisor stack when Tony Tebbys driver is at it's deepest level of subroutine calls and register saves. Add to that subroutine calls and register saves made by the operating system, before calling the device driver, it could push it over the edge. (maybe an LBYTES neti_x command uses more supervisor stack space than LBYTES#. So LBYTES# gets away with it)

Martin Head

[Martin_Head]

A quick update to the above post, for anyone interested.

I tried to reduce the load on the system stack, but it did not help.

I then tried adding some "Print" statements in the driver to track how far the code got before it fell over.

LBYTES neti_x suddenly started to work! So I then started removing the "Print" statements one at a time to try to isolate the problem. But to cut a long story short, it was not very conclusive.

What I am thinking now is. Is QPC2 too fast for windows, as in LBYTES neti_x opens the IP stream, then tries to read from it before Windows has finished setting it up in the background. And the "Print" statements are slowing the driver down giving Windows a chance to catch up.

Maybe I need to stick a delay loop in the Open routine.

Martin Head

[tofro]

Martin,

interesting you got it going so far, even with some "tricks".

However, even if the underlying "device" - in your case a socket, obviously - Is not able to deliver the data requested, it shouldn't crash. The driver routines should come back with the number of bytes read (0) and a "not complete" in d0. This, in turn, should cause QDOS to come back later and retry the same operation. Does your driver handle that cause?

Regards,
Tobias

[Martin_Head]

However, even if the underlying "device" - in your case a socket, obviously - Is not able to deliver the data requested, it shouldn't crash. The driver routines should come back with the number of bytes read (0) and a "not complete" in d0. This, in turn, should cause QDOS to come back later and retry the same operation. Does your driver handle that cause?

I've not got around to trying out the delay loop idea yet. When I say that QPC2 crashes, I don't know if it actually crashes , or if it's just got stuck waiting for an IP call which never completes, or times out.

My routine to read a data packet isn't just one IP recv call, its three, involving loops. First it loops "Peeking" the data until at least 2 bytes are available (the packet length). Then it loops "Peeking" again, waiting for for at least the length of the data packet to be available. Then it actually reads the data packet.

I could probably do with some kind of keyboard read in these loops to enable you to escape from the routine, (something on the things to do list)

The IP recv calls currently have a 2 minute timeout set, but the driver does not come back after the 2 minutes.

As you say "The driver routines should come back with the number of bytes read (0) and a "not complete" in d0." This is one of the odd things I have noticed, When the LBYTES command succeeds. My "Print" statements I added display "Entering I/O, Calling read packet, Entering I/O, Calling read packet" then the cursor reappears. Which is correct as the data uses 2 packets.

However when the LBYTES command fails I get "Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet, Entering I/O, Calling read packet," then it stops with no cursor.

It looks like the IP recv calls are coming back with "Not Complete", and SMSQ/E is retrying 8 or 9 times. But I would have thought if SMSQ/E had given up it would have returned with an error, rather than just hang.

Martin Head