NETDEV WATCHDOG: eth0: transmit timed out

Recently I encountered a problem with my desktop computer running Ubuntu 7.10 Gutsy Gibbon.

After some time network would die and the only way to bring it back was to restart the computer.
Nothing worked:

• NetworkManager
  
• sudo /etc/init.d/networking restart
• reloading kernel modules

Looking into the /var/syslog I found this:

Dec 7 08:09:36 moonbiter kernel: [ 3751.383550] NETDEV WATCHDOG: eth0: transmit timed out
Dec 7 08:09:36 moonbiter kernel: [ 3751.383626] eth0: Transmit timed out, status 0003, PHY status 786d, resetting...
Dec 7 08:09:36 moonbiter kernel: [ 3751.383939] eth0: link up, 100Mbps, full-duplex, lpa 0x45E1

I have two ethernet cards: one builtin and one in a PCI slot (lspci comes in handy):

00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 7c)
05:07.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet (rev 10)

Googling around provided with a lot of contradicting solutions. None of them worked. Finally, on one of the forums I found an advice to check IRQs. Indeed this could have been the problem. I recently bought nVidia GeForce 7300 GT and to fit it in I had to move my Realtek ethernet adapter to another PCI slot.

Looking into /proc/interrupts I found:

CPU0
0: 46332 IO-APIC-edge timer
1: 155 IO-APIC-edge i8042
6: 5 IO-APIC-edge floppy
7: 0 IO-APIC-edge parport0
8: 0 IO-APIC-edge rtc
9: 0 IO-APIC-fasteoi acpi
12: 1617 IO-APIC-edge i8042
14: 15664 IO-APIC-edge ide0
15: 1242 IO-APIC-edge ide1
17: 717 IO-APIC-fasteoi eth1, HDA Intel
20: 0 IO-APIC-fasteoi uhci_hcd:usb1
21: 0 IO-APIC-fasteoi sata_via, uhci_hcd:usb3, ehci_hcd:usb5
22: 0 IO-APIC-fasteoi uhci_hcd:usb2
23: 13202 IO-APIC-fasteoi uhci_hcd:usb4, eth0
24: 13183 IO-APIC-fasteoi nvidia
NMI: 0
LOC: 46223
ERR: 0

And in /var/syslog

Dec 7 06:59:06 moonbiter kernel: [ 1637.162744] irq 23: nobody cared (try booting with the "irqpoll" option)
Dec 7 06:59:06 moonbiter kernel: [ 1637.162748]
Dec 7 06:59:06 moonbiter kernel: [ 1637.162749] Call Trace:
Dec 7 06:59:06 moonbiter kernel: [ 1637.162751] [__report_bad_irq+30/128] __report_bad_irq+0x1e/0x80
Dec 7 06:59:06 moonbiter kernel: [ 1637.162770] [note_interrupt+643/704] note_interrupt+0x283/0x2c0
Dec 7 06:59:06 moonbiter kernel: [ 1637.162777] [handle_fasteoi_irq+221/272] handle_fasteoi_irq+0xdd/0x110
Dec 7 06:59:06 moonbiter kernel: [ 1637.162897] [_end+129724926/2130332920] :nvidia:_nv003707rm+0x1f/0x27
Dec 7 06:59:06 moonbiter kernel: [ 1637.162904] [do_IRQ+123/256] do_IRQ+0x7b/0x100
Dec 7 06:59:06 moonbiter kernel: [ 1637.162909] [ret_from_intr+0/10] ret_from_intr+0x0/0xa
Dec 7 06:59:06 moonbiter kernel: [ 1637.162914] [pci_conf1_read+0/272] pci_conf1_read+0x0/0x110
Dec 7 06:59:06 moonbiter kernel: [ 1637.162921] [__do_softirq+84/224] __do_softirq+0x54/0xe0
Dec 7 06:59:06 moonbiter kernel: [ 1637.162929] [call_softirq+28/48] call_softirq+0x1c/0x30
Dec 7 06:59:06 moonbiter kernel: [ 1637.162933] [do_softirq+53/144] do_softirq+0x35/0x90
Dec 7 06:59:06 moonbiter kernel: [ 1637.162937] [do_IRQ+128/256] do_IRQ+0x80/0x100
Dec 7 06:59:06 moonbiter kernel: [ 1637.162942] [ret_from_intr+0/10] ret_from_intr+0x0/0xa
Dec 7 06:59:06 moonbiter kernel: [ 1637.162944] [_end+127707073/2130332920] :processor:acpi_processor_idle+0x25f/0x456
Dec 7 06:59:06 moonbiter kernel: [ 1637.162964] [_end+127707063/2130332920] :processor:acpi_processor_idle+0x255/0x456
Dec 7 06:59:06 moonbiter kernel: [ 1637.162971] [_end+127706466/2130332920] :processor:acpi_processor_idle+0x0/0x456
Dec 7 06:59:06 moonbiter kernel: [ 1637.162976] [cpu_idle+112/192] cpu_idle+0x70/0xc0
Dec 7 06:59:06 moonbiter kernel: [ 1637.162982] [start_kernel+645/784] start_kernel+0x285/0x310
Dec 7 06:59:06 moonbiter kernel: [ 1637.162987] [x86_64_start_kernel+286/352] _sinittext+0x11e/0x160
Dec 7 06:59:06 moonbiter kernel: [ 1637.162991]
Dec 7 06:59:06 moonbiter kernel: [ 1637.162992] handlers:
Dec 7 06:59:06 moonbiter kernel: [ 1637.162994] [_end+128326184/2130332920] (usb_hcd_irq+0x0/0x60 [usbcore])
Dec 7 06:59:06 moonbiter kernel: [ 1637.163010] [_end+128535048/2130332920] (rhine_interrupt+0x0/0xc70 [via_rhine])
Dec 7 06:59:06 moonbiter kernel: [ 1637.163017] Disabling IRQ #23

So irq assigned to eth0 was disabled ...
Following advice from the forum I added "noapic" boot option to my /boot/grub/menu.lst
And now network is solid stable again :D.

Just to see how /proc/interrupts looks now:

CPU0
0: 145441 XT-PIC-XT timer
1: 1186 XT-PIC-XT i8042
2: 0 XT-PIC-XT cascade
4: 0 XT-PIC-XT uhci_hcd:usb3, ehci_hcd:usb5
5: 24510 XT-PIC-XT uhci_hcd:usb1, uhci_hcd:usb4, eth0
6: 5 XT-PIC-XT floppy
7: 47352 XT-PIC-XT parport0
8: 0 XT-PIC-XT rtc
9: 0 XT-PIC-XT acpi
10: 48019 XT-PIC-XT nvidia
11: 740 XT-PIC-XT sata_via, uhci_hcd:usb2, eth1, HDA Intel
12: 7125 XT-PIC-XT i8042
14: 51693 XT-PIC-XT ide0
15: 5002 XT-PIC-XT ide1
NMI: 0
LOC: 145342
ERR: 4

I'm not too happy with this solution but it works...

Sniffing the Google Desktop for Linux traffic.

I promised to present how to sniff communication between clients and server of Google Desktop for Linux.

So I'll be using SockSniffer scripts I prepared and wrote about [1], [2].

$ tar xvzf SockSniffer.0.2.tar.gz
$ mkdir jail
$ cd jail
$ $HOME/SockSniffer.0.2/makejail.sh
$ sudo $HOME/SockSniffer.0.2/mountjail.sh
$ sudo chroot $PWD
$ su my_user_name
$ cd
$ gdlinux

No we should have two Google Desktop icons in taskbar.

In chrooted console:

$ cd .google/desktop
$ rm a1_sock

In Another console:

$ mkdir socksniff_logs
$ cd socksniff_logs
$ $HOME/SockSniffer.0.2/socksniff $HOME/.google/desktop/a1_sock $HOME/jail$HOME/.google/desktop/a1_sock

Now use the Google Desktop for a while.
Close it.
Kill the socksniff process (e.g. using ^c).

In socksniff console:

$ $HOME/SockSniffer.0.2/sockmerge

Open $HOME/socksniff_logs/dumps/merge.pcap in wireshark.

That's it. Now we can analyze the traffic.

And here's an example dump.

SockSniffer v.0.2

New version of SockSniffer (v0.2):

• makejail.sh and mountjail.sh scripts added.
  These scripts are used to prepare chroot jail for client to be sniffed.
  They are prepared with GoogleDesktop in mind.
• Added some comments in the code.
• The socksniff script was corrected: The -n option didn't work for server traffic.

SockSniffer.0.2.tar.gz

Unix socket sniffer

There are a lot of tools for peeking into communication that is performed using the network sockets:

• wireshark
• tcpdump
• snort

to just name a few.

But I didn't find any program that could perform this treat for the unix sockets. I found only one comment on some mailing list that someone hacked the Linux kernel to perform this (most probably altered the read syscall) but no code whatsoever.

I didn't want to start hacking the kernel as my only experience with it was compiling it during short time when I run Gentoo on my desktop. Then it occurred to me: Why not use the chroot?

Idea is quite simple. A Server and a client communicate via the unix socket, which is a named socket and is located somewhere in the file system. We run the server in standard way. For the client we create a chroot environment. We run the client in the chroot jail. Our sniffer program connects two sockets: the standard one with the one in chroot jail. While transferring the traffic it can dump it into a log file :-D.

So I wrote an app that performs this: SockSniffer ;-).
In the packet there are four files:

• socksniff - the sniffer, it will output the log files into $PWD/dumps directory.
• sockmerge - this program will merge log files in $PWD/dumps directory and output it in libpcap format which is readable for wireshark.
• client.pl and server.pl - client and server scripts to test the sniffer.

All those scripts are written in Perl. Output of socksniff are hex dumps of the traffic. One file per "packet". To use the sockmerge script wireshark (or ethereal) needs to be installed.

In coming days I will post a guide how to sniff traffic generated by the GoogleDesktop for Linux.

Final Fantasy X - expert Sphere Grid

I recently bought an used PlayStation 2 console.

I must say that for a device that was designed over 7 years ago it has still astonishing graphics. And now can be bought very cheap :-D.

Of course I bought Final Fantasy X - to have finally a PS2 and to not play in one (actually all ;-) ) of the FFs would be a sin :-D.

In FFX characters are advanced by spending Ability Points and various spheres on a Sphere Grid. I searched for a diagram of the grid so that I could sit down with a pen and plan paths for my characters. I found this one. It has unfortunately a black background, so there is no point in printing it. I took gimp for a spin and modified it. Bellow is the result :-D.

Restoring grub in MBR

There is a lot of stuff about this.

But of course none worked for me :-D.
So I decided to post a solution that worked ;-).

1. Boot with any live CD (I've done it with Knoppix 3.x and Ubuntu)
2. Prepare directory for mounting:
   $ sudo mkdir /mnt/ubuntu
3. Mount the root (/) partition of ubuntu e.g.:
   $ sudo mount /dev/hda5 /mnt/ubuntu
4. Mount proc and dev filesystems on mounted partition:
   $ mount -o bind /dev /mnt/ubuntu/dev
   $ mount -o bind /proc /mnt/ubuntu/proc
5. If /boot is on seperate partition mount it:
   $ sudo mount /dev/hda4 /mnt/ubuntu/boot
   
6. Chroot the mounted partition:
   $ chroot /mnt/ubuntu
7. Populate /etc/mtab:
   $ sudo cp /proc/mounts /etc/mtab
8. $ sudo grub-install /dev/hda
   
9. Exit the shell
10. Reboot

Hope that it'll help someone ;-)

Poland and Ukraine will host Euro 2012

Today UEFA announced that UEFA Euro 2012 championships will be organized by Poland and Ukraine.

I must admit that I was quite sure that the winner will be Italy. Actually I was not alone, lots of people here in Poland didn't believe that we could win.

BUT WE WON!!!

GO POLAND AND UKRAINE!!!

Here is a link to official site of the organizers.

Coffee

Recently we bought a Moka Express pot.

I must admit that it brews the best coffee I tasted at home. It's very easy to operate and hopefully to clean ;-). Brewing is also quite quick :-D

Our pot is actually a cheep imitation of the original "Alfonso Bialetti's Moka Express". Here's how it looks:

Just Married

On 14 of April 2007 I got married :-D.

My wife's name is Katarzyna (Cathrine) but everybody calls her Kasia.

We are a couple for five and a half years now.
We studied together Physics on Warsaw Technical University.

The wedding went marvelous. We arrived in stylish white "Warszawa"[pl] (which is actually licensed Pobeda) . We had beautiful weather - it was sunny and warm but not hot. Cathy looked stunning, and our guest say that my morning suit looked great ;-). Church[pl] in which wedding took place has unforgettable atmosphere, especially inside.

Our wedding party was also a great success :-D. The ball room[pl] was beautifully decorated, food was delicious and looked great ;-). But the best was the band[pl]. They simply rocked ]:->. Well they didn't forget about standards, so our aunts and uncles were also very happy ;-).

So to make long story short:
I'm a happy husband now :-D.

Please visit our website[pl]. When we'll get some pictures from the wedding we'll put them there.

Unlock the Gnome keyring upon login - Xubuntu

As many people already noticed NetworManager uses gnome-keyring to store password for different wireless networks. This is already true for Dapper Drake.

In Edgy and Feisty more and more programs start to use gnome-keyring for the same purpose. To name a few: Evolution, Gajim.

This is quite annoying because one has to unlock the keyring database every time one logs in. On my desktop with freshly installed Feisty I use only Gajim from those three. Which makes storing my google talk password virtually useless.

Fortunately there is a solution: pam-keyring.
Here you'll find how to install it for Feisty.

Shortly how its done:

sudo apt-get install libpam-keyring
sudo echo "@include common-pamkeyring" >> /etc/pam.d/gdm

Unfortunately it didn't work on my Xubutu at first.
Apparently one has to disable "Launch Gnome services on startup" to make it work.

What seems to happen is that xfce4-session lunches new instance of gnome-keyring-deamon without checking if there is already a running one. This new instance of course does not have the keyring database unlocked by pam-keyring, hence we get the password popup.

I've filled a bug report we'll see what happens...