VIF weirdness

View: New views
14 Messages — Rating Filter:   Alert me  

VIF weirdness

by Page, Jeremy :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

I am having a great deal of difficulty getting my filer’s 10g interfaces connected to a pair of 3750E switches I am trying to use for my storage network’s backbone. I can inconsistently ping other objects on the network (there really is nothing there yet, we are just getting  started, but I do have a host and an RLM card just for troubleshooting purposes), but they cannot ping each other.  From a remote host (172.1.2.5)  I can ping everything but array1. If I bring down either of the two stacked switches (which brings down one port of each VIF member pairs) everything works.

 

array01> ping 172.1.0.1

172.1.0.1 is alive

array01> ping 172.1.0.2

array01> ping 172.1.0.4

ping: wrote 172.1.0.4 64 chars, error=Host is down

ping: wrote 172.1.0.4 64 chars, error=Host is down

array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]: Local NIS group update successful.

array01> ping 172.1.2.5

172.1.2.5 is alive

 

array02> ping 172.1.0.1

array02> ping 172.1.0.4

172.1.0.4 is alive

array02> ping 172.1.0.4

172.1.0.4 is alive

array02> ping 172.1.0.250

172.1.0.250 is alive

array02> ping 172.1.2.5 

no answer from 172.1.2.5

 

 

array01> ifconfig -a

e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol full

e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol full

e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160

        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1

        ether 00:00:00:00:00:00 (VIA Provider)

lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255

        partner 10.28.17.214 (not in use)

        ether 02:a0:98:08:22:b7 (Enabled virtual interface)

tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500

        inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255

        partner 172.1.0.2 (not in use)

        ether 02:a0:98:08:22:b6 (Enabled virtual interface)

        nfo enabled

array02>ifconfig –a

e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol full

e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol full

e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160

        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1

        ether 00:00:00:00:00:00 (VIA Provider)

lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255

        partner 10.28.17.213 (not in use)

        ether 02:a0:98:0c:1a:37 (Enabled virtual interface)

tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500

        inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255

        partner 172.1.0.1 (not in use)

        ether 02:a0:98:0c:1a:36 (Enabled virtual interface)

        nfo enabled

This message (including any attachments) contains confidential
and/or proprietary information intended only for the addressee.
Any unauthorized disclosure, copying, distribution or reliance on
the contents of this information is strictly prohibited and may
constitute a violation of law. If you are not the intended
recipient, please notify the sender immediately by responding to
this e-mail, and delete the message from your system. If you
have any questions about this e-mail please notify the sender
immediately.


Parent Message unknown RE: VIF weirdness

by Page, Jeremy :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

They are 10 gig cards, they are only in  a VIF with themselves (no other cards). It’s a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g connections. The 2 1000 are on our production network as lan0 VIF for CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on the storage only network (no routing etc, private for storage devices and the hosts using them only).

 

They are plugged into stacked 3750e boxes, I tried putting them on the same switch just to make sure it was not a stacking thing with no real change.

 

The oddness is that occasionally a single packet will get through, and if I bring the interfaces up and down it may change who can get to what.

 


From: Webster, Stetson [mailto:Stetson.Webster@...]
Sent: Thursday, May 15, 2008 2:13 PM
To: Page, Jeremy; toasters@...
Subject: RE: VIF weirdness

 

Are these 10g TOE cards?  Are they in a VIF with other 'non-TOE' cards?

 

Stetson M. Webster
Onsite Professional Services Engineer
PS - North Amer. - East

NetApp
919.250.0052 Mobile
Stetson.Webster@...www.netapp.com

 

 


From: Page, Jeremy [mailto:jeremy.page@...]
Sent: Thursday, May 15, 2008 1:32 PM
To: toasters@...
Subject: VIF weirdness

I am having a great deal of difficulty getting my filer’s 10g interfaces connected to a pair of 3750E switches I am trying to use for my storage network’s backbone. I can inconsistently ping other objects on the network (there really is nothing there yet, we are just getting  started, but I do have a host and an RLM card just for troubleshooting purposes), but they cannot ping each other.  >From a remote host (172.1.2.5)  I can ping everything but array1. If I bring down either of the two stacked switches (which brings down one port of each VIF member pairs) everything works.

 

array01> ping 172.1.0.1

172.1.0.1 is alive

array01> ping 172.1.0.2

array01> ping 172.1.0.4

ping: wrote 172.1.0.4 64 chars, error=Host is down

ping: wrote 172.1.0.4 64 chars, error=Host is down

array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]: Local NIS group update successful.

array01> ping 172.1.2.5

172.1.2.5 is alive

 

array02> ping 172.1.0.1

array02> ping 172.1.0.4

172.1.0.4 is alive

array02> ping 172.1.0.4

172.1.0.4 is alive

array02> ping 172.1.0.250

172.1.0.250 is alive

array02> ping 172.1.2.5 

no answer from 172.1.2.5

 

 

array01> ifconfig -a

e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol full

e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol full

e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160

        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1

        ether 00:00:00:00:00:00 (VIA Provider)

lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255

        partner 10.28.17.214 (not in use)

        ether 02:a0:98:08:22:b7 (Enabled virtual interface)

tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500

        inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255

        partner 172.1.0.2 (not in use)

        ether 02:a0:98:08:22:b6 (Enabled virtual interface)

        nfo enabled

array02>ifconfig –a

e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol full

e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full

        trunked lan0

e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol full

e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full

        trunked tgif

lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160

        inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1

        ether 00:00:00:00:00:00 (VIA Provider)

lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

        inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255

        partner 10.28.17.213 (not in use)

        ether 02:a0:98:0c:1a:37 (Enabled virtual interface)

tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500

        inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255

        partner 172.1.0.1 (not in use)

        ether 02:a0:98:0c:1a:36 (Enabled virtual interface)

        nfo enabled

This message (including any attachments) contains confidential
and/or proprietary information intended only for the addressee.
Any unauthorized disclosure, copying, distribution or reliance on
the contents of this information is strictly prohibited and may
constitute a violation of law. If you are not the intended
recipient, please notify the sender immediately by responding to
this e-mail, and delete the message from your system. If you
have any questions about this e-mail please notify the sender
immediately.

This message (including any attachments) contains confidential
and/or proprietary information intended only for the addressee.
Any unauthorized disclosure, copying, distribution or reliance on
the contents of this information is strictly prohibited and may
constitute a violation of law. If you are not the intended
recipient, please notify the sender immediately by responding to
this e-mail, and delete the message from your system. If you
have any questions about this e-mail please notify the sender
immediately.



Re: VIF weirdness

by tmac-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

How about a "sysconfig -v 2" from the filers
and a "show running-config " that shows at a minimum:
the port config of the 10-gig stuff and the port-channel definitions.

--tmac

On Thu, May 15, 2008 at 2:19 PM, Page, Jeremy <jeremy.page@...> wrote:

> They are 10 gig cards, they are only in  a VIF with themselves (no other
> cards). It's a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g
> connections. The 2 1000 are on our production network as lan0 VIF for
> CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on the
> storage only network (no routing etc, private for storage devices and the
> hosts using them only).
>
>
>
> They are plugged into stacked 3750e boxes, I tried putting them on the same
> switch just to make sure it was not a stacking thing with no real change.
>
>
>
> The oddness is that occasionally a single packet will get through, and if I
> bring the interfaces up and down it may change who can get to what.
>
>
>
> ________________________________
>
> From: Webster, Stetson [mailto:Stetson.Webster@...]
> Sent: Thursday, May 15, 2008 2:13 PM
> To: Page, Jeremy; toasters@...
> Subject: RE: VIF weirdness
>
>
>
> Are these 10g TOE cards?  Are they in a VIF with other 'non-TOE' cards?
>
>
>
> Stetson M. Webster
> Onsite Professional Services Engineer
> PS - North Amer. - East
>
> NetApp
> 919.250.0052 Mobile
> Stetson.Webster@...
> www.netapp.com
>
>
>
>
>
> ________________________________
>
> From: Page, Jeremy [mailto:jeremy.page@...]
> Sent: Thursday, May 15, 2008 1:32 PM
> To: toasters@...
> Subject: VIF weirdness
>
> I am having a great deal of difficulty getting my filer's 10g interfaces
> connected to a pair of 3750E switches I am trying to use for my storage
> network's backbone. I can inconsistently ping other objects on the network
> (there really is nothing there yet, we are just getting  started, but I do
> have a host and an RLM card just for troubleshooting purposes), but they
> cannot ping each other.  >From a remote host (172.1.2.5)  I can ping
> everything but array1. If I bring down either of the two stacked switches
> (which brings down one port of each VIF member pairs) everything works.
>
>
>
> array01> ping 172.1.0.1
>
> 172.1.0.1 is alive
>
> array01> ping 172.1.0.2
>
> array01> ping 172.1.0.4
>
> ping: wrote 172.1.0.4 64 chars, error=Host is down
>
> ping: wrote 172.1.0.4 64 chars, error=Host is down
>
> array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]: Local NIS
> group update successful.
>
> array01> ping 172.1.2.5
>
> 172.1.2.5 is alive
>
>
>
> array02> ping 172.1.0.1
>
> array02> ping 172.1.0.4
>
> 172.1.0.4 is alive
>
> array02> ping 172.1.0.4
>
> 172.1.0.4 is alive
>
> array02> ping 172.1.0.250
>
> 172.1.0.250 is alive
>
> array02> ping 172.1.2.5
>
> no answer from 172.1.2.5
>
>
>
>
>
> array01> ifconfig -a
>
> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol full
>
> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol full
>
> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>
>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>
>         ether 00:00:00:00:00:00 (VIA Provider)
>
> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255
>
>         partner 10.28.17.214 (not in use)
>
>         ether 02:a0:98:08:22:b7 (Enabled virtual interface)
>
> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500
>
>         inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255
>
>         partner 172.1.0.2 (not in use)
>
>         ether 02:a0:98:08:22:b6 (Enabled virtual interface)
>
>         nfo enabled
>
> array02>ifconfig –a
>
> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol full
>
> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol full
>
> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>
>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>
>         ether 00:00:00:00:00:00 (VIA Provider)
>
> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255
>
>         partner 10.28.17.213 (not in use)
>
>         ether 02:a0:98:0c:1a:37 (Enabled virtual interface)
>
> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS> mtu 1500
>
>         inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255
>
>         partner 172.1.0.1 (not in use)
>
>         ether 02:a0:98:0c:1a:36 (Enabled virtual interface)
>
>         nfo enabled
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law. If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system. If you
> have any questions about this e-mail please notify the sender
> immediately.
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law. If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system. If you
> have any questions about this e-mail please notify the sender
> immediately.
>



--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant


RE: VIF weirdness

by Page, Jeremy :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I *think* the problem may be isolated to array1. I've tried setting them
to single mode instead of multi (LACP is not supposed to work on a
stacked switch).


array01> sysconfig -v 2
        slot 2: Dual TOE-10G Ethernet Controller (T320E-XFP)
                Device Type:        CT-31-1
                Version Number:
T3-SRAM1.1.0-BR1040-20-C0-FW4.6.0-DR03
                Serial Number:      PT4107009
                e2a MAC Address:    00:07:43:05:16:98
(auto-10g_sr-fd-up)
                e2b MAC Address:    00:07:43:05:16:99
(auto-10g_sr-fd-up)


**switch config (short form)**
version 12.2
no aaa new-model
switch 1 provision ws-c3750e-24td
switch 2 provision ws-c3750e-24td
interface TenGigabitEthernet1/0/1
 switchport mode access
 channel-group 1 mode on
!
interface TenGigabitEthernet1/0/2
 switchport mode access
 channel-group 2 mode on
interface TenGigabitEthernet2/0/1
 switchport mode access
 channel-group 1 mode on
!
interface TenGigabitEthernet2/0/2
 switchport mode access
 channel-group 2 mode on
!
interface Vlan1
 ip address 172.1.0.250 255.255.0.0

-----Original Message-----
From: tmac [mailto:tmacmd@...]
Sent: Thursday, May 15, 2008 3:46 PM
To: Page, Jeremy
Cc: toasters@...
Subject: Re: VIF weirdness

How about a "sysconfig -v 2" from the filers
and a "show running-config " that shows at a minimum:
the port config of the 10-gig stuff and the port-channel definitions.

--tmac

On Thu, May 15, 2008 at 2:19 PM, Page, Jeremy <jeremy.page@...>
wrote:
> They are 10 gig cards, they are only in  a VIF with themselves (no
other
> cards). It's a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g
> connections. The 2 1000 are on our production network as lan0 VIF for
> CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on
the
> storage only network (no routing etc, private for storage devices and
the
> hosts using them only).
>
>
>
> They are plugged into stacked 3750e boxes, I tried putting them on the
same
> switch just to make sure it was not a stacking thing with no real
change.
>
>
>
> The oddness is that occasionally a single packet will get through, and
if I

> bring the interfaces up and down it may change who can get to what.
>
>
>
> ________________________________
>
> From: Webster, Stetson [mailto:Stetson.Webster@...]
> Sent: Thursday, May 15, 2008 2:13 PM
> To: Page, Jeremy; toasters@...
> Subject: RE: VIF weirdness
>
>
>
> Are these 10g TOE cards?  Are they in a VIF with other 'non-TOE'
cards?

>
>
>
> Stetson M. Webster
> Onsite Professional Services Engineer
> PS - North Amer. - East
>
> NetApp
> 919.250.0052 Mobile
> Stetson.Webster@...
> www.netapp.com
>
>
>
>
>
> ________________________________
>
> From: Page, Jeremy [mailto:jeremy.page@...]
> Sent: Thursday, May 15, 2008 1:32 PM
> To: toasters@...
> Subject: VIF weirdness
>
> I am having a great deal of difficulty getting my filer's 10g
interfaces
> connected to a pair of 3750E switches I am trying to use for my
storage
> network's backbone. I can inconsistently ping other objects on the
network
> (there really is nothing there yet, we are just getting  started, but
I do
> have a host and an RLM card just for troubleshooting purposes), but
they
> cannot ping each other.  >From a remote host (172.1.2.5)  I can ping
> everything but array1. If I bring down either of the two stacked
switches
> (which brings down one port of each VIF member pairs) everything
works.

>
>
>
> array01> ping 172.1.0.1
>
> 172.1.0.1 is alive
>
> array01> ping 172.1.0.2
>
> array01> ping 172.1.0.4
>
> ping: wrote 172.1.0.4 64 chars, error=Host is down
>
> ping: wrote 172.1.0.4 64 chars, error=Host is down
>
> array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]:
Local NIS

> group update successful.
>
> array01> ping 172.1.2.5
>
> 172.1.2.5 is alive
>
>
>
> array02> ping 172.1.0.1
>
> array02> ping 172.1.0.4
>
> 172.1.0.4 is alive
>
> array02> ping 172.1.0.4
>
> 172.1.0.4 is alive
>
> array02> ping 172.1.0.250
>
> 172.1.0.250 is alive
>
> array02> ping 172.1.2.5
>
> no answer from 172.1.2.5
>
>
>
>
>
> array01> ifconfig -a
>
> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol
full

>
> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol
full

>
> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>
>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>
>         ether 00:00:00:00:00:00 (VIA Provider)
>
> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255
>
>         partner 10.28.17.214 (not in use)
>
>         ether 02:a0:98:08:22:b7 (Enabled virtual interface)
>
> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
mtu 1500

>
>         inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255
>
>         partner 172.1.0.2 (not in use)
>
>         ether 02:a0:98:08:22:b6 (Enabled virtual interface)
>
>         nfo enabled
>
> array02>ifconfig -a
>
> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol
full

>
> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>
>         trunked lan0
>
> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol
full

>
> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>
>         trunked tgif
>
> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>
>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>
>         ether 00:00:00:00:00:00 (VIA Provider)
>
> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>
>         inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255
>
>         partner 10.28.17.213 (not in use)
>
>         ether 02:a0:98:0c:1a:37 (Enabled virtual interface)
>
> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
mtu 1500

>
>         inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255
>
>         partner 172.1.0.1 (not in use)
>
>         ether 02:a0:98:0c:1a:36 (Enabled virtual interface)
>
>         nfo enabled
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law. If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system. If you
> have any questions about this e-mail please notify the sender
> immediately.
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law. If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system. If you
> have any questions about this e-mail please notify the sender
> immediately.
>



--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant


This message (including any attachments) contains confidential
and/or proprietary information intended only for the addressee.  
Any unauthorized disclosure, copying, distribution or reliance on
the contents of this information is strictly prohibited and may
constitute a violation of law.  If you are not the intended
recipient, please notify the sender immediately by responding to
this e-mail, and delete the message from your system.  If you
have any questions about this e-mail please notify the sender
immediately.


Re: VIF weirdness

by tmac-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Notes aboutof 10GbE TOE NIC limitations : The 10GbE TOE NIC cards
have a number of limitations. They include:
◆ Multimode vif limited to two (2) 10GbE TOE NICs
◆ LACP not supported with 10GbE TOE NICs
◆ TOE functionality disabled on 10GbE NIC in vif


Do you need a
switchport access vlan ###
in the interface lines?

Here is the config I use for vifs on my switch:

interface Port-Channel7
 description netapp-vif
 switchport
 switchport access vlan 239
 switchport mode access
 no ip address
 flowcontrol receive on
 flowcontrol send on
 spanning-tree portfast
!
interface GigabitEthernet7/42
 description filer-e9
 switchport
 switchport access vlan 239
 switchport mode access
 no ip address
 flowcontrol receive on
 flowcontrol send on
 spanning-tree portfast
 channel-group 7 mode active
!

In your case, with 10-gig, the only change should be:
channel-group 7 mode on
(which you are already doing)


Also, make sure your filer is setup correctly (not using LACP for 10Gig)

vif create multi tgif -b mac e2a r3b

On Thu, May 15, 2008 at 3:55 PM, Page, Jeremy <jeremy.page@...> wrote:

> I *think* the problem may be isolated to array1. I've tried setting them
> to single mode instead of multi (LACP is not supposed to work on a
> stacked switch).
>
>
> array01> sysconfig -v 2
>        slot 2: Dual TOE-10G Ethernet Controller (T320E-XFP)
>                Device Type:        CT-31-1
>                Version Number:
> T3-SRAM1.1.0-BR1040-20-C0-FW4.6.0-DR03
>                Serial Number:      PT4107009
>                e2a MAC Address:    00:07:43:05:16:98
> (auto-10g_sr-fd-up)
>                e2b MAC Address:    00:07:43:05:16:99
> (auto-10g_sr-fd-up)
>
>
> **switch config (short form)**
> version 12.2
> no aaa new-model
> switch 1 provision ws-c3750e-24td
> switch 2 provision ws-c3750e-24td
> interface TenGigabitEthernet1/0/1
>  switchport mode access
>  channel-group 1 mode on
> !
> interface TenGigabitEthernet1/0/2
>  switchport mode access
>  channel-group 2 mode on
> interface TenGigabitEthernet2/0/1
>  switchport mode access
>  channel-group 1 mode on
> !
> interface TenGigabitEthernet2/0/2
>  switchport mode access
>  channel-group 2 mode on
> !
> interface Vlan1
>  ip address 172.1.0.250 255.255.0.0
>
> -----Original Message-----
> From: tmac [mailto:tmacmd@...]
> Sent: Thursday, May 15, 2008 3:46 PM
> To: Page, Jeremy
> Cc: toasters@...
> Subject: Re: VIF weirdness
>
> How about a "sysconfig -v 2" from the filers
> and a "show running-config " that shows at a minimum:
> the port config of the 10-gig stuff and the port-channel definitions.
>
> --tmac
>
> On Thu, May 15, 2008 at 2:19 PM, Page, Jeremy <jeremy.page@...>
> wrote:
>> They are 10 gig cards, they are only in  a VIF with themselves (no
> other
>> cards). It's a 3070 with 2 copper 100/1000 2 FC gig and 2 FC 10g
>> connections. The 2 1000 are on our production network as lan0 VIF for
>> CIFS/NFS, the fibre gig is not used and the two 10g FC are VIF tgif on
> the
>> storage only network (no routing etc, private for storage devices and
> the
>> hosts using them only).
>>
>>
>>
>> They are plugged into stacked 3750e boxes, I tried putting them on the
> same
>> switch just to make sure it was not a stacking thing with no real
> change.
>>
>>
>>
>> The oddness is that occasionally a single packet will get through, and
> if I
>> bring the interfaces up and down it may change who can get to what.
>>
>>
>>
>> ________________________________
>>
>> From: Webster, Stetson [mailto:Stetson.Webster@...]
>> Sent: Thursday, May 15, 2008 2:13 PM
>> To: Page, Jeremy; toasters@...
>> Subject: RE: VIF weirdness
>>
>>
>>
>> Are these 10g TOE cards?  Are they in a VIF with other 'non-TOE'
> cards?
>>
>>
>>
>> Stetson M. Webster
>> Onsite Professional Services Engineer
>> PS - North Amer. - East
>>
>> NetApp
>> 919.250.0052 Mobile
>> Stetson.Webster@...
>> www.netapp.com
>>
>>
>>
>>
>>
>> ________________________________
>>
>> From: Page, Jeremy [mailto:jeremy.page@...]
>> Sent: Thursday, May 15, 2008 1:32 PM
>> To: toasters@...
>> Subject: VIF weirdness
>>
>> I am having a great deal of difficulty getting my filer's 10g
> interfaces
>> connected to a pair of 3750E switches I am trying to use for my
> storage
>> network's backbone. I can inconsistently ping other objects on the
> network
>> (there really is nothing there yet, we are just getting  started, but
> I do
>> have a host and an RLM card just for troubleshooting purposes), but
> they
>> cannot ping each other.  >From a remote host (172.1.2.5)  I can ping
>> everything but array1. If I bring down either of the two stacked
> switches
>> (which brings down one port of each VIF member pairs) everything
> works.
>>
>>
>>
>> array01> ping 172.1.0.1
>>
>> 172.1.0.1 is alive
>>
>> array01> ping 172.1.0.2
>>
>> array01> ping 172.1.0.4
>>
>> ping: wrote 172.1.0.4 64 chars, error=Host is down
>>
>> ping: wrote 172.1.0.4 64 chars, error=Host is down
>>
>> array01> Thu May 15 13:28:33 EDT [gvr-array01: nis_worker_0:info]:
> Local NIS
>> group update successful.
>>
>> array01> ping 172.1.2.5
>>
>> 172.1.2.5 is alive
>>
>>
>>
>> array02> ping 172.1.0.1
>>
>> array02> ping 172.1.0.4
>>
>> 172.1.0.4 is alive
>>
>> array02> ping 172.1.0.4
>>
>> 172.1.0.4 is alive
>>
>> array02> ping 172.1.0.250
>>
>> 172.1.0.250 is alive
>>
>> array02> ping 172.1.2.5
>>
>> no answer from 172.1.2.5
>>
>>
>>
>>
>>
>> array01> ifconfig -a
>>
>> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:08:22:b6 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b7 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:08:22:b4 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:08:22:b6 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>>
>>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>>
>>         ether 00:00:00:00:00:00 (VIA Provider)
>>
>> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         inet 10.28.17.213 netmask 0xffffff00 broadcast 10.28.17.255
>>
>>         partner 10.28.17.214 (not in use)
>>
>>         ether 02:a0:98:08:22:b7 (Enabled virtual interface)
>>
>> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
> mtu 1500
>>
>>         inet 172.1.0.1 netmask 0xffff0000 broadcast 172.1.255.255
>>
>>         partner 172.1.0.2 (not in use)
>>
>>         ether 02:a0:98:08:22:b6 (Enabled virtual interface)
>>
>>         nfo enabled
>>
>> array02>ifconfig -a
>>
>> e0a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:0c:1a:36 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e0c: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:37 (auto-1000t-fd-up) flowcontrol full
>>
>>         trunked lan0
>>
>> e0d: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 00:a0:98:0c:1a:34 (auto-unknown-cfg_down) flowcontrol
> full
>>
>> e2a: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> e2b: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         ether 02:a0:98:0c:1a:36 (auto-10g_sr-fd-up) flowcontrol full
>>
>>         trunked tgif
>>
>> lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
>>
>>         inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
>>
>>         ether 00:00:00:00:00:00 (VIA Provider)
>>
>> lan0: flags=948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
>>
>>         inet 10.28.17.214 netmask 0xffffff00 broadcast 10.28.17.255
>>
>>         partner 10.28.17.213 (not in use)
>>
>>         ether 02:a0:98:0c:1a:37 (Enabled virtual interface)
>>
>> tgif: flags=4948043<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM,NOWINS>
> mtu 1500
>>
>>         inet 172.1.0.2 netmask 0xffff0000 broadcast 172.1.255.255
>>
>>         partner 172.1.0.1 (not in use)
>>
>>         ether 02:a0:98:0c:1a:36 (Enabled virtual interface)
>>
>>         nfo enabled
>>
>> This message (including any attachments) contains confidential
>> and/or proprietary information intended only for the addressee.
>> Any unauthorized disclosure, copying, distribution or reliance on
>> the contents of this information is strictly prohibited and may
>> constitute a violation of law. If you are not the intended
>> recipient, please notify the sender immediately by responding to
>> this e-mail, and delete the message from your system. If you
>> have any questions about this e-mail please notify the sender
>> immediately.
>>
>> This message (including any attachments) contains confidential
>> and/or proprietary information intended only for the addressee.
>> Any unauthorized disclosure, copying, distribution or reliance on
>> the contents of this information is strictly prohibited and may
>> constitute a violation of law. If you are not the intended
>> recipient, please notify the sender immediately by responding to
>> this e-mail, and delete the message from your system. If you
>> have any questions about this e-mail please notify the sender
>> immediately.
>>
>
>
>
> --
> --tmac
>
> RedHat Certified Engineer #804006984323821 (RHEL4)
> RedHat Certified Engineer #805007643429572 (RHEL5)
>
> Principal Consultant
>
>
> This message (including any attachments) contains confidential
> and/or proprietary information intended only for the addressee.
> Any unauthorized disclosure, copying, distribution or reliance on
> the contents of this information is strictly prohibited and may
> constitute a violation of law.  If you are not the intended
> recipient, please notify the sender immediately by responding to
> this e-mail, and delete the message from your system.  If you
> have any questions about this e-mail please notify the sender
> immediately.
>



--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant

RE: VIF weirdness

by Glenn Walker :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.

Only time I have seen stuff like that is when it’s in multimode on the filer, and etherchannel is not properly enabled on the switch.

 


From: owner-toasters@... [mailto:owner-toasters@...] On Behalf Of Page, Jeremy
Sent: Thursday, May 15, 2008 1:32 PM
To: toasters@...
Subject: VIF weirdness

 

I am having a great deal of difficulty getting my filer’s 10g interfaces connected to a pair of 3750E switches I am trying to use for my storage network’s backbone. I can inconsistently ping other objects on the network (there really is nothing there yet, we are just getting  started, but I do have a host and an RLM card just for troubleshooting purposes), but they cannot ping each other.  From a remote host (172.1.2.5)  I can ping everything but array1. If I bring down either of the two stacked switches (which brings down one port of each VIF member pairs) everything works.

 

array01> ping 172.1.0.1

172.1.0.1 is alive

array01> ping 172.1.0.2

array01> ping 172.1.0.4