Panic

Strange thing happen.

0. Start your FreeBSD single-user.
1. Run /etc/netstart so your network is up.
2. Run /etc/rc.d/sshd start so you have sshd
3. Use ssh to login localhost, then su(1) to root
4. Type “init q”

Then init(8) would crash with signal 11. Yes, it was killed. As soon as kernel found this, it would panic with “Going nowhere without my init!”.

Now I got (partially) the point:

• A newly fork’ed thread grabs Giant
• Subsequently, the mutex procedure will call turnstile_wait
• However, the td->td_sched->ke_runq is NULL (means that some part of the new thread is not initialized completely)
• So boom! panic.

Interestingly this does not happen in !PREEMPTION case.

Now my question are:

1. who forks?
2. is it the child? the parent?
3. why the initialization is not completed with a wrong result?

I have done several experiments/asserts to ensure that the panic is related to ULE+PREEMPTION case only.

ULE: no panic
4BSD: no panic
4BSD+PREEMPTION: no panic

My latest progress is that I found that sched_switch in ULE for the threaded case, can decide whether subsequent call would panic. Investigating.

Finally I caught the kernel panic caused by some strange bugs found in thread scheduling. Investigating the crashdump.

I have ported Matt’s patch to 4-STABLE however Matthew Seaman reports that there is still panic, while different, after the patch and I believe I should port Matt’s VM post-morti diagnostic tools to HEAD.

Before reporting this to the list, I would first gather enough information.

Alu’s server boxes is suffering in continued reboot when server is loaded. Because he is supporting FreeBSD by utilizing 5.3-RC1 in his production environment (and it’s moderately loaded as it is connected to the ChinaNet Backbone), and donated many devices for dedicated services of the FreeBSD China Community, I think we must help him to solve these out.

I have plugged a new kernel configuration on his box in order to obtain a (potential) crashdump and figure out what is happening. Maybe we will get it tomorrow.