Everything posted by guru
-
Capture DHCP traffic using Wireshark
Step One (Select Interface) Open Wireshark and go to (Capture -> Interfaces) Determine which Ethernet device you are using to connect to the network. You can determine which one is being used by the number of packets sent/received. NOTE: I’m using the one called eth0, which is my virtual network card. Step Two (Choose Options) Click the options button on the device being used to bring up the capture options menu. Uncheck the capture packets in promiscuous mode option to only see traffic that is sent and received to this network card. Step Three (Start Capture) Click the start button to begin capturing network traffic. Now Wireshark is capturing all of the traffic that is sent and received by the network card. Step Four (Filter) We are only interested with the DHCP traffic, so on the display filter type (bootp.option.type == 53) and click apply. The DHCP Release resulted from me typing Linux: sudo dhclient -r Windows: ipconfig /release Mac: sudo ifconfig en1 down (list interfaces: networksetup -listnetworkserviceorder) The DHCP Discover, Offer, Request, and ACK resulted from me typing Linux: sudo dhclient eth0 Windows: ipconfig /renew Mac: sudo ifconfig en1 up If you look at the DHCP Response you can see the DHCP Scope Options that have been sent to the client from the DHCP Server DHCP (Dynamic Host Configuration Protocol) Discovery DHCP uses the same two ports assigned by IANA for BOOTP: destination UDP port 67 for sending data to the server, and UDP port 68 for data to the client. DHCP operations fall into four basic phases: IP discovery, IP lease offer, IP request, and IP lease acknowledgment. DHCP clients and servers on the same subnet communicate via UDP broadcasts, initially. If the client and server are on different subnets, a DHCP Helper or DHCP Relay Agent may be used. Clients requesting renewal of an existing lease may communicate directly via UDP unicast, since the client already has an established IP address at that point. The client broadcasts messages on the physical subnet to discover available DHCP servers. Network administrators can configure a local router (or IP Helper Address on a switch) to forward DHCP packets to a DHCP server which resides on a different subnet. This client implementation creates a User Datagram Protocol (UDP) packet with the broadcast destination of 255.255.255.255 or the specific subnet broadcast address. A DHCP client can also request its last-known IP address (in the example below, 192.168.1.100). If the client remains connected to a network for which this IP is valid, the server may grant the request. Otherwise, it depends whether the server is set up as authoritative or not. An authoritative server will deny the request, making the client ask for a new IP address immediately. A non-authoritative server simply ignores the request, leading to an implementation-dependent timeout for the client to give up on the request and ask for a new IP address. DHCP Offer When a DHCP server receives an IP lease request from a client, it reserves an IP address for the client and extends an IP lease offer by sending a DHCPOFFER message to the client. This message contains the client's MAC address, the IP address that the server is offering, the subnet mask, the lease duration, and the IP address of the DHCP server making the offer. The server determines the configuration based on the client's hardware address as specified in the CHADDR (Client Hardware Address) field. Here the server, 192.168.1.1, specifies the client's IP address in the YIADDR (Your IP Address) field. DHCP Request In response to the DHCP offer, the client replies with a DHCP request, broadcast to the server, requesting the offered address. A client can receive DHCP offers from multiple servers, but it will accept only one DHCP offer. Based on required server identification option in the request and broadcast messaging, servers are informed whose offer the client has accepted. When other DHCP servers receive this message, they withdraw any offers that they might have made to the client and return the offered address to the pool of available addresses. DHCP Acknowledgement When the DHCP server receives the DHCPREQUEST message from the client, the configuration process enters its final phase. The acknowledgement phase involves sending a DHCPACK packet to the client. This packet includes the lease duration and any other configuration information that the client might have requested. At this point, the IP configuration process is completed. After the client obtains an IP address, the client may use the Address Resolution Protocol (ARP) to prevent IP conflicts caused by overlapping address pools of DHCP servers. The protocol expects the DHCP client to configure its network interface with the negotiated parameters.
-
TTL expired in transit
When I ping 1.2.3.4... I get this. Pinging 1.2.3.4 with 32 bytes of data: Reply from 1.1.1.1: TTL expired in transit. Reply from 1.1.1.1: TTL expired in transit. Reply from 1.1.1.1: TTL expired in transit. Reply from 1.1.1.1: TTL expired in transit. Ping statistics for 1.2.3.4: Packets: Sent = 4, Received = 4, Lost = 0 (0 Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms The TTL (Time To Live) value determines the maximum amount of time an IP packet may live in the network without reaching its destination. It is effectively a bound on the number of routers an IP packet may pass through before being discarded. This message indicates that the TTL expired in transit. Increase the TTL value using the -i parameter with the ping command. Most computers today initialize the TTL value of outgoing IP Packets 128 or higher. If you ever see a reply above with a "TTL=5" (or some other low TTL number) this tells you that the computer being pinged should most likely have its default TTL value increased. Otherwise, anyone trying to communicate with the computer that is at a hop count higher than the TTL will not be able to communicate with the computer. You could also be experiencing issues at a firewall with ICMP (ping) being blocked/dropped or issues with Network Address Translations (NAT) not working or setup properly. In this situation the NAT was removed If you find that ICMP is being blocked then you can use nmap which can use TCP. So instead of using ICMP, which is a layer3 (network), the TCP or layer4 (transport) layer is utilized. The default behavior of NMAP is to do both an ICMP ping sweep (the usual kind of ping) and a TCP port 80 ACK ping sweep. If an admin is logging these this will be fairly characteristic of NMAP. This behavior can be changed in several ways. The easiest way is, of course, to simply turn off ping sweeps with -P0. If you want to do a standard ICMP ping sweep use -PI. If you are trying to get through a firewall, though, ICMP pings will likely be blocked and using packet filtering ICMP pings can even be dropped at the host. To get around this NMAP tries to do a TCP "ping" to see if a host is up. By default it sends an ACK to port 80 and expects to see a RST from that port if the host is up. To do only this scan and not the ICMP ping scan use -PT. To specify a different port than port 80 to scan for specify it immediately afterwards, e.g. -PT32523 will ACK ping port 32523. Picking a random high-numbered port in this way may work *much* better than the default NMAP behavior of ACK pinging port 80. This is because many packet filter rules are setup to let through all packets to high numbered ports with the ACK bit set, but sites may filter port 80 on every machine other than their publically accessable webservers. You can also do both an ICMP ping scan and an ACK scan to a high numbered port with, e.g. -PB32523. However, if a site has a really, really intelligent firewall that recognizes that your ACK packet isn't part of an ongoing TCP connection it might be smart enough to block it. For that reason, you may get better results with a TCP SYN sweep with -PS. In this case, scanning a high-numbered port will probably not work, and instead you need to pick a port which is likely to get through a firewall. Port 80 is not a bad pick, but something like ssh (port 22) may be better.
-
What is DHCP?
Dynamic Host Configuration Protocol (DHCP) is a network protocol that enables a server to automatically assign an IP address to a computer from a defined range of numbers (i.e., a scope) configured for a given network. DHCP assigns an IP address when a system is started, for example: A user turns on a computer with a DHCP client. The client computer sends a broadcast request (called a DISCOVER or DHCPDISCOVER), looking for a DHCP server to answer. The router directs the DISCOVER packet to the correct DHCP server. UDP Src=0.0.0.0 sPort=68 Dest=255.255.255.255 dPort=67 The server receives the DISCOVER packet. Based on availability and usage policies set on the server, the server determines an appropriate address (if any) to give to the client. The server then temporarily reserves that address for the client and sends back to the client an OFFER (or DHCPOFFER) packet, with that address information. The server also configures the client's DNS servers, WINS servers, NTP servers, and sometimes other services as well. UDP Src=192.168.1.1 sPort=67 Dest=255.255.255.255 dPort=68 The client sends a REQUEST (or DHCPREQUEST) packet, letting the server know that it intends to use the address. UDP Src=0.0.0.0 sPort=68 Dest=255.255.255.255 dPort=67 The server sends an ACK (or DHCPACK) packet, confirming that the client has a been given a lease on the address for a server-specified period of time. UDP Src=192.168.1.1 sPort=67 Dest=255.255.255.255 dPort=68
-
Anycast described
Many hardware and software solutions exist to implement redundancy and load balancing for hosted services. One way to implement load balancing and redundancy is to use anycast, defined in RFC 1546. In an anycast setup, multiple hosts share the same IP address. This address is announced through a routing protocol, so that packets sent to the anycast address will be routed to the (network topology wise) closest host. A paper from Cisco provides a good background on anycast. Another, more theoretical paper is available from IBM. Because anycast relies only on a routing protocol, no additional hardware or software is needed to implement it. Since it relies on inherently dynamic routing protocols (such as OSPF or BGP) to decide which host packets are routed to, it is generally only useful for protocols that require very little state, such as DNS. According to some, in practice this instability is not significant enough to prevent anycast from being used for TCP-based services. Adds redundancy and load balancing to connectionless client/server services and improve availability and possibly latency. Anycast is a communication model (network service) for IPv4 and IPv6. As originally described in RFC 1546, "Host Anycast Service," the purpose of anycast is to assign an identical anycast address to a group of geographically distributed nodes. IP datagrams approach the nearest destination node in the set of available destination nodes, based on the unicast routing measure of distance transparent to the clients. The network (routing system) decides where to guide the client request. An IPv4 anycast address is distinguishable from a unicast address because they are allocated from a special reserved range. This is different in IPv6. The real-world applications of anycast I am aware of are limited to DNS root server concepts and Protocol Independent Multicast (PIM) rendezvous points, stateless protocols in general. Anycast is usually implemented by using Border Gateway Protocol (BGP) to simultaneously announce the same destination IP address range from many different places on the network. This results in packets addressed to destination addresses in this range being routed to the "nearest" point on the net announcing the given destination IP address. In the past, anycast was suited to connectionless protocols (generally built on UDP), rather than connection-oriented protocols such as TCP that keep their own state. However, there are many cases where TCP anycast is now used. With TCP anycast, there are cases where the receiver selected for any given source may change from time to time as optimal routes change, silently breaking any conversations that may be in progress at the time. These conditions are typically referred to as a "pop switch". To correct for this issue, there have been proprietary advancements within custom IP stacks which allow for healing of stateful protocols where it is required. For this reason, anycast is generally used as a way to provide high availability and load balancing for stateless services such as access to replicated data; for example, DNS service is a distributed service over multiple geographically dispersed servers. Unicast addressing uses a one-to-one association between destination address and network endpoint: each destination address uniquely identifies a single receiver endpoint. Multicast addressing uses a one-to-unique many association, datagrams are routed from a single sender to multiple selected endpoints simultaneously in a single transmission. Broadcast addressing uses a one-to-many association, datagrams are routed from a single sender to multiple endpoints simultaneously in a single transmission. The network automatically replicates datagrams as needed for all network segments (links) that contain an eligible receiver. Anycast addressing routes datagrams to a single member of a group of potential receivers that are all identified by the same destination address. This is a one-to-nearest association. Cisco Router Configuration ip sla 101 dns anycast.example.com name-server 10.10.10.1 frequency 30 ip sla schedule 101 life forever start-time now ! track 101 ip sla 101 ! ip route 10.0.0.1 255.255.255.255 10.10.10.1 track 101 Here is the IP route on the router: router# show ip route 10.0.0.1 Routing entry for 10.0.0.1/32 Known via "static", distance 1, metric 0 Redistributing via eigrp 1234 Advertised by eigrp 1234 route-map STATIC-TO-EIGRP bgp 1234 Routing Descriptor Blocks: * 10.10.10.1 Route metric is 0, traffic share count is 1 Then you can see that this same address is also available from multiple locations: router# show ip eigrp topology 10.0.0.1/32 EIGRP-IPv4 Topology Entry for AS(1234)/ID(10.9.9.1) for 10.0.0.1/32 State is Passive, Query origin flag is 1, 1 Successor(s), FD is 2562560 Descriptor Blocks: 10.10.10.1, from Rstatic, Send flag is 0x0 ... 10.8.8.1 (Vlan20), from 10.6.6.1, Send flag is 0x0 ... 10.7.7.1 (Vlan30), from 10.4.4.1, Send flag is 0x0 On the Unix server I have the following network interfaces setup: eth0 Link encap:Ethernet HWaddr 00:15:17:A6:25:97 inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0 lo:1 Link encap:Local Loopback inet addr:10.0.0.1 Mask:255.255.255.255 To summarize the whole setup. The router does a DNS query to the DNS server that is directly connected to it every 30 seconds. If the DNS query succeeds the static router stays in the table. If the test fails the route is withdrawn. If a DNS query is sent to 10.0.0.1 the router will process this by sending the query to the IP address the static route points to. The DNS server accepts the query on the management interface, then passes it to the lo:1 interface for processing. Depending on where you are at you automatically get routed to the closest server: dj@thezah:~$ traceroute 10.0.0.1 traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets 1 l3-core-vl7.nts.example.com (10.50.1.46) 0.309 ms 0.338 ms 0.381 ms 2 anycast.ip.example.com (10.0.0.1) 0.202 ms 0.195 ms 0.180 ms dj@hosangit:~$ sudo traceroute 10.0.0.1 traceroute to 10.0.0.1 (10.0.0.1), 64 hops max, 52 byte packets 1 nts-desk120-brook.nts.example.com (10.50.120.125) 0 ms 0 ms 0 ms 2 anycast.ip.example.com (10.0.0.1) 0 ms 0 ms 0 ms The best thing about this setup is: If a server fails you automatically fail over to the next closest server. This way the client does not have to deal with DNS times outs. Depending on your location you are automatically routed to the closest server. This will help with DNS response time. It is not that hard to setup. Nothing special is needed either the router or the server.
-
Troubleshoot DHCP
This is an attempt to help troubleshoot DHCP issues. First let me attempt at explaining how DHCP works to better understand how to troubleshoot it. Let's talk about DHCP at the home which is more simple. Your router typically does everything to providing wireless signal for you to connect to as well as route traffic to your internet provider via a cable modem or other device that connects you to the internet. The router also provides DHCP to all the devices in your home like your phone, iPad, Smart TV, Apple TV, computer, Playstation etc.. So what happens when you connect a device to your WiFi or plug directly into your router with a network cable? The new device calls out and says, "I want an IP address" and the DHCP server see's the request and assigns them the next available IP address. You can confirm this on your device... Windows - in the search bar type cmd and command prompt should pop up, click on that then enter ipconfig /all and press enter. This will give you your ip address as well as a few other very helpful bits of information like the servers that resolve a name to an IP called DNS servers, also you'll see your default gateway which is typically the ip address of your router (the way out of your network to the internet). Mac/Linux - you can run ifconfig | egrep 'mtu|ether|inet' and it will show you the IP address next to inet and the ether is the hardware address What if you don't have an IP address? Here is a couple suggestions Typically a reboot of the system is the default action since we know that rebooting a device will request an IP address if it doesn't have one. In Windows you can type: ipconfig /renew and it will request a new IP address from the DHCP server
-
BIG-IP GTM change script status
I put together this script that checks the status of everything prior to the change and then run it after the change and then download the two log files and open in something like VSCode and do a compare to see if the change you did on the GTM changed the status of something. It has helped me and I hope it helps you. I'm sure there is something better but this is what I know how to do. ** USE AT YOUR OWN RISK ** #!/bin/bash ## Author: Cowboy Denny ## Last Modified: 2024.05.06 ## RUN THIS FILE ON THE GTM in the /var/tmp directory ## GTM: Change Verify BEFORE & AFTER change deployment ## RUN with: bash gtmchgverify.sh unset totalgtmwips; unset totalgtmwipsa; unset totalgtmwipsu; unset totalgtmwipso; unset totalgtmwipsd; unset totalgtmwipsun; unset totalgtmpools; unset totalgtmpoolsa; unset totalgtmpoolsu; unset totalgtmpoolso; unset totalgtmpoolsd; unset totalgtmpoolsun; unset totalgtmdc; unset totalgtmdca; unset totalgtmdcu; unset totalgtmdco; unset totalgtmdcd; unset totalgtmdcun; unset totalgtmiquery; unset totalgtmiqueryc; unset totalgtmpp; unset totalgtmppa; unset totalgtmppu; unset totalgtmppo; unset totalgtmppd; unset totalgtmppun; unset totalgtmsrv; unset totalgtmsrva; unset totalgtmsrvu; unset totalgtmsrvo; unset totalgtmsrvd; unset totalgtmsrvun; unset totalgtmlist; unset totalgtmlista; unset totalgtmlistu; unset totalgtmlisto; unset totalgtmlistd; unset totalgtmlistun; unset gtmchgfilename; clear echo "GTM: Data Gathering has started... this takes about 60-90 seconds" gtmchgfilename=/var/tmp/$(echo $HOSTNAME | cut -d'.' -f1)-$(date +%Y%m%d_%H%M)-STATUS.log; date > $gtmchgfilename; echo "SERVER specific INFO for hostname...." >> $gtmchgfilename; tmsh list sys global-settings hostname | grep hostname >> $gtmchgfilename; echo "F5 running version...." >> $gtmchgfilename; tmsh -q show sys software status >> $gtmchgfilename; echo "Master Key is..." >> $gtmchgfilename; f5mku -K >> $gtmchgfilename; echo ""; echo "1/3 PATIENCE while we gather statistics" ##WIDE IPS export totalgtmwips=$(tmsh -q -c 'cd / ; show gtm wideip recursive' | grep -c 'Gtm::WideIp'); export totalgtmwipsa=$(tmsh -q -c 'cd / ; show gtm wideip recursive' | egrep 'Gtm::WideIp|Availability'| grep -c 'available'); export totalgtmwipsu=$(tmsh -q -c 'cd / ; show gtm wideip recursive' | egrep 'Gtm::WideIp|Availability'| grep -c 'unavailable'); export totalgtmwipso=$(tmsh -q -c 'cd / ; show gtm wideip recursive' | egrep 'Gtm::WideIp|Availability'| grep -c 'offline'); export totalgtmwipsd=$(tmsh -q -c 'cd / ; show gtm wideip recursive' | egrep 'Gtm::WideIp|Availability'| grep -c 'disabled'); export totalgtmwipsun=$(tmsh -q -c 'cd / ; show gtm wideip recursive' | egrep 'Gtm::WideIp|Availability'| grep -c 'unknown'); ##POOLS export totalgtmpools=$(tmsh -q -c 'cd / ; show gtm pool recursive' | grep -c 'Gtm::Pool'); export totalgtmpoolsa=$(tmsh -q -c 'cd / ; show gtm pool recursive' | egrep 'Gtm::Pool|Availability'| grep -c 'available'); export totalgtmpoolsu=$(tmsh -q -c 'cd / ; show gtm pool recursive' | egrep 'Gtm::Pool|Availability'| grep -c 'unavailable'); export totalgtmpoolso=$(tmsh -q -c 'cd / ; show gtm pool recursive' | egrep 'Gtm::Pool|Availability'| grep -c 'offline'); export totalgtmpoolsd=$(tmsh -q -c 'cd / ; show gtm pool recursive' | egrep 'Gtm::Pool|Availability'| grep -c 'disabled'); export totalgtmpoolsun=$(tmsh -q -c 'cd / ; show gtm pool recursive' | egrep 'Gtm::Pool|Availability'| grep -c 'unknown'); #DATACENTERS export totalgtmdc=$(tmsh show gtm datacenter | grep -c 'Gtm::Datacenter'); export totalgtmdca=$(tmsh show gtm datacenter | egrep 'Gtm::Datacenter|Availability'| grep -c 'available'); export totalgtmdcu=$(tmsh show gtm datacenter | egrep 'Gtm::Datacenter|Availability'| grep -c 'unavailable'); export totalgtmdco=$(tmsh show gtm datacenter | egrep 'Gtm::Datacenter|Availability'| grep -c 'offline'); export totalgtmdcd=$(tmsh show gtm datacenter | egrep 'Gtm::Datacenter|Availability'| grep -c 'disabled'); export totalgtmdcun=$(tmsh show gtm datacenter | egrep 'Gtm::Datacenter|Availability'| grep -c 'unknown'); #IQUERY export totalgtmiquery=$(tmsh show gtm iquery | grep -c 'Gtm::IQuery'); export totalgtmiqueryc=$(tmsh show gtm iquery | egrep 'Gtm::IQuery|State'| grep -c 'connected'); #PROBERPOOLS export totalgtmpp=$(tmsh show gtm prober-pool | grep -c 'Gtm::Prober Pool'); export totalgtmppa=$(tmsh show gtm prober-pool | egrep 'Gtm::Prober Pool|Availability'| grep -c 'available'); export totalgtmppu=$(tmsh show gtm prober-pool | egrep 'Gtm::Prober Pool|Availability'| grep -c 'unavailable'); export totalgtmppo=$(tmsh show gtm prober-pool | egrep 'Gtm::Prober Pool|Availability'| grep -c 'offline'); export totalgtmppd=$(tmsh show gtm prober-pool | egrep 'Gtm::Prober Pool|Availability'| grep -c 'disabled'); export totalgtmppun=$(tmsh show gtm prober-pool | egrep 'Gtm::Prober Pool|Availability'| grep -c 'unknown'); #SERVERS export totalgtmsrv=$(tmsh show gtm server | grep -c 'Gtm::Server'); export totalgtmsrva=$(tmsh show gtm server | egrep 'Gtm::Server|Availability'| grep -c 'available'); export totalgtmsrvu=$(tmsh show gtm server | egrep 'Gtm::Server|Availability'| grep -c 'unavailable'); export totalgtmsrvo=$(tmsh show gtm server | egrep 'Gtm::Server|Availability'| grep -c 'offline'); export totalgtmsrvd=$(tmsh show gtm server | egrep 'Gtm::Server|Availability'| grep -c 'disabled'); export totalgtmsrvun=$(tmsh show gtm server | egrep 'Gtm::Server|Availability'| grep -c 'unknown'); #LISTENERS export totalgtmlist=$(tmsh show gtm listener | grep -c 'Gtm::Listener'); export totalgtmlista=$(tmsh show gtm listener | egrep 'Gtm::Listener|Availability'| grep -c 'available'); export totalgtmlistu=$(tmsh show gtm listener | egrep 'Gtm::Listener|Availability'| grep -c 'unavailable'); export totalgtmlisto=$(tmsh show gtm listener | egrep 'Gtm::Listener|Availability'| grep -c 'offline'); export totalgtmlistd=$(tmsh show gtm listener | egrep 'Gtm::Listener|Availability'| grep -c 'disabled'); export totalgtmlistun=$(tmsh show gtm listener | egrep 'Gtm::Listener|Availability'| grep -c 'unknown'); echo "2/3 PATIENCE adding the stats gathered to $gtmchgfilename" echo "**************************" >> $gtmchgfilename; echo "* STATS for $HOSTNAME *" >> $gtmchgfilename; echo "***********************" >> $gtmchgfilename; echo " Total GTMWideIPs:$totalgtmwips - Available:$totalgtmwipsa - Unavailable:$totalgtmwipsu - Offline:$totalgtmwipso - Disabled:$totalgtmwipsd - Unknown:$totalgtmwipsun" >> $gtmchgfilename; echo " Total GTMPools:$totalgtmpools - Available:$totalgtmwipsa - Unavailable:$totalgtmwipsu - Offline:$totalgtmwipso - Disabled:$totalgtmwipsd - Unknown:$totalgtmwipsun" >> $gtmchgfilename; echo " Total GTMDataCenters:$totalgtmdc - Available:$totalgtmdca - Unavailable:$totalgtmdcsu - Offline:$totalgtmdco - Disabled:$totalgtmdcd - Unknown:$totalgtmdcun" >> $gtmchgfilename; echo " Total iQueryStats:$totalgtmiquery - Connected:$totalgtmiqueryc" >> $gtmchgfilename; echo " Total ProberPools:$totalgtmpp - Available:$totalgtmppa - Unavailable:$totalgtmppu - Offline:$totalgtmppo - Disabled:$totalgtmppd - Unknown:$totalgtmppun" >> $gtmchgfilename; echo " Total Servers:$totalgtmsrv - Available:$totalgtmsrva - Unavailable:$totalgtmsrvu - Offline:$totalgtmsrvo - Disabled:$totalgtmsrvd - Unknown:$totalgtmsrvun" >> $gtmchgfilename; echo " Total Listeners:$totalgtmlist - Available:$totalgtmlista - Unavailable:$totalgtmlistu - Offline:$totalgtmlisto - Disabled:$totalgtmlistd - Unknown:$totalgtmlistun" >> $gtmchgfilename; echo "3/3 PATIENCE final step is to gather details to help troubleshoot to $gtmchgfilename" echo "*************************************************************************************************************" echo "now exporting all GTM WideIPs to file" echo "********" >> $gtmchgfilename; echo "WIDE-IPs" >> $gtmchgfilename; echo "********" >> $gtmchgfilename; tmsh -q -c 'cd / ; show gtm wideip recursive' | egrep 'Gtm::WideIp|Availability :|State|Reason' >> $gtmchgfilename; echo "*************************************************************************************************************" echo "now exporting all GTM Pools to file" echo "********" >> $gtmchgfilename; echo "*POOLsv*" >> $gtmchgfilename; echo "********" >> $gtmchgfilename; tmsh -q -c 'cd / ; show gtm pool recursive' | egrep 'Gtm::Pool|Availability|State|Reason' >> $gtmchgfilename; echo "*************************************************************************************************************" echo "now exporting all DataCenters to file" echo "***********" >> $gtmchgfilename; echo "DATACENTERS" >> $gtmchgfilename; echo "***********" >> $gtmchgfilename; tmsh show gtm datacenter | egrep 'Gtm::Datacenter|Availability|State|Reason' >> $gtmchgfilename; echo "*************************************************************************************************************" echo "now exporting all iQuery to file" echo "***********" >> $gtmchgfilename; echo "**IQUERY***" >> $gtmchgfilename; echo "***********" >> $gtmchgfilename; tmsh show gtm iquery | egrep 'Gtm::IQuery|Server|Data Center|State|version' >> $gtmchgfilename; echo "*************************************************************************************************************" echo "now exporting all ProberPool to file" echo "***********" >> $gtmchgfilename; echo "PROBER-POOL" >> $gtmchgfilename; echo "***********" >> $gtmchgfilename; tmsh show gtm prober-pool | egrep 'Gtm::Prober Pool|Availability|State|Reason' >> $gtmchgfilename; echo "*************************************************************************************************************" echo "now exporting all Servers to file" echo "***********" >> $gtmchgfilename; echo "**SERVER***" >> $gtmchgfilename; echo "***********" >> $gtmchgfilename; tmsh show gtm server | egrep 'Gtm::Server|Availability|State|Reason' >> $gtmchgfilename; echo "*************************************************************************************************************" echo "now exporting all Listeners to file" echo "***********" >> $gtmchgfilename; echo "**LISTENER*" >> $gtmchgfilename; echo "***********" >> $gtmchgfilename; tmsh show gtm listener | egrep 'Gtm::Listener|Availability|State|Reason' >> $gtmchgfilename; echo "****************************************************************************************************************" echo "" date >> $gtmchgfilename; echo "Done with $HOSTNAME change data gathering. NOTE for more info look at output file: $gtmchgfilename " unset totalgtmwips; unset totalgtmwipsa; unset totalgtmwipsu; unset totalgtmwipso; unset totalgtmwipsd; unset totalgtmwipsun; unset totalgtmpools; unset totalgtmpoolsa; unset totalgtmpoolsu; unset totalgtmpoolso; unset totalgtmpoolsd; unset totalgtmpoolsun; unset totalgtmdc; unset totalgtmdca; unset totalgtmdcu; unset totalgtmdco; unset totalgtmdcd; unset totalgtmdcun; unset totalgtmiquery; unset totalgtmiqueryc; unset totalgtmpp; unset totalgtmppa; unset totalgtmppu; unset totalgtmppo; unset totalgtmppd; unset totalgtmppun; unset totalgtmsrv; unset totalgtmsrva; unset totalgtmsrvu; unset totalgtmsrvo; unset totalgtmsrvd; unset totalgtmsrvun; unset totalgtmlist; unset totalgtmlista; unset totalgtmlistu; unset totalgtmlisto; unset totalgtmlistd; unset totalgtmlistun; unset gtmchgfilename; echo "Now exiting" exit
-
Looking up hostname from URL
I have a task of locating a server based off of URL So what my thought process is would be to first find the IP address ping url.com Then if it is a Windows server, try and utilize nbtstat -A ip.address or nbtstat -a computername Tried tracert url.com then connected to last hop/switch sh arp | incl ip.address sh mac-address-table address mac.address (may want to lookup mac address for vendor) if it shows up on a trunk or multiple ports, see if its going to another switch sh cdp nei (look for the port that the sh mac-address-table address result included) Also tried nslookup ip.address Also downloaded nmap and tried os detection but it couldn't identify the mac address nmap -sVC -O -T4 url.com Anyone have any other idea's for looking up information for a server in an intranet (internal LAN) I use nmap also and have no idea on how else to do that
-
CNAME versus A record
So whats the difference? hosangit.example.com. CNAME mywiseguys.example.com. mywiseguys.example.com. A 192.168.2.23 When an A record lookup for hosangit.example.com is done, the resolver will see a CNAME record and restart the checking at mywiseguys.example.com and will then return 192.168.2.23.
-
How Do I Redirect My Site Using DNS?
I'm moving my website from my school machine (blah.mypc.com) to my own domain (myname.com). I parked myname.com a long time ago and set it as a simple redirect to blah.mypc.com following the instructions at Hostmonster, where I registered it. Now I'm going to convert myname.com to a full site, and I want my personal address blah.mypc.com to disappear as a machine and be nothing but a simple dns redirect to myname.com. What exactly do I want to ask my school tech people to do (in terms they will recognize)? Do I want a CNAME record created that will permanently redirect all incoming traffic from blah.mypc.com to myname.com? (There is no ftp or email traffic at this address, only http.) If not CNAME, what is it that I should request? The existing machine blah.mypc.com is one physical machine with a unique I.P. It's been my experience that a CNAME for your old site to your new domain name is a really good way to go about it. Here's why: Pro's: you can leave it there forever and forget about it with little drain on the university resources. (having them serve a refresh web-page or a redirect is probably more likely to get screwed up next time they diddle with their webserver) you can later change the DNS of your new machine around (if you get new IP addresses, for instance) and the university name points to your new domain name regardless of the IP address you can treat the traffic from people who are trying to go to your old name differently. (set up apache on your new machine to serve that virtual host differently, for instance to redirect to your new site with a notice or something of the change) you can serve your site entirely with no redirects or refreshes, which some people claim affect spiders/search engine's rankings of your site. Use apache's ServerName directive to correct the hostname that the browser requested. Con's if people send mail to you at your old university machine, it may have problems because your MX record may end up being pointed to a CNAME record, which is not strictly acceptable by standards. you may have to set up your new web server to either have this extra virtual host (and treat it just like your new domain name) or to serve pages for any hostname which points to your new IP address (often called a default virtual host, which you can't do sometimes if you're just one virtual host on your hosting company's machine) There are probably more arguments on each side, but I think that putting in a CNAME record for your old hostname pointing to your new one is the best way to go about it. Example: In their dns zone files they have a record similar to the following blah.mypc.com. IN A 1.2.2.2 To do a dns redirect using a CNAME record they need to change blah.mypc.com. IN A 1.2.2.2 to blah.mypc.com. IN CNAME myname.com. The main drawback is that the old URL " http://blah.mypc.com " will appear in the browser's address bar. Anyway to redirect the queries and keep the links unchanged? I don't want to loose the google queries or static links that people have from other sites So if I have http://blah.mypc.com/forums/mywiseguys-is-awesome.html I would like it to redirect to http://mysite.com/forums/mywiseguys-is-awesome.html What's the best way to do this? If you have access to your .htaccess file you may want to give this a try Options +FollowSymLinks RewriteEngine On RewriteBase / RewriteCond %{HTTP_HOST} !^www\.mysite\.com$ RewriteRule ^(.*)$ http://www.mysite.com/$1 It seems to work pretty good for me but I just learned that this is a restriction of BIND and not F5 BigIP or even Cisco The best way to analyze a URL is in four parts scheme (http, https, etc.) hostname (mywiseguys.com) path (/forums/gallery/) query (#entry620) BIND DNS can address only the hostname portion of this where a load balancer pretty much doesn't have much for limitations.
-
Where is my hosts file
I want to add an IP address to a DNS name so I don't have to rely on DNS to do the resolution for me but I can not seem to locate the hosts file when I do a search on my machine. Any ideas? Sometimes doing a search will not find the hosts file. Typically the hosts file is located on a windows machine at the following location: %SystemRoot%\system32\drivers\etc\ So you can click on Start - Run and type: See if the file is there in explorer run: explorer %SystemRoot%\system32\drivers\etc\ To edit the file run: notepad %SystemRoot%\system32\drivers\etc\hosts to edit the file If you aren't seeing the file in that location then the administrator may have moved the location of the hosts file. No worries, you can find the new location by looking in the registry at: \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\DataBasePath
-
DNS Troubleshooting
First check the domain using nslookup nslookup set querytype=ns mywiseguys.com Next check the alias or hostname set querytype=a branch.mywiseguys.com Using nslookup interactively Enter nslookup at the command line. The nslookup prompt appears. View the current options by entering set all. Change any desired options by entering set option. Issue nslookup commands. Enter exit to leave nslookup. For a list of sample commands, see ``nslookup interactive commands''. For a list of options, see ``nslookup interactive options''. nslookup interactive commands These sample commands are available from the nslookup shell: volga Return the IP address of volga. 172.16.118.1 Return the name matching the IP address you enter. set querytype=ns Set the query type to the Name Server record. Future queries of names and IP addresses return the NS record from that host. set querytype=a Restore the query type to the Address record. server server Make server the default server that is queried. nslookup interactive options Here are the commonly used options of nslookup. For a complete list, see the manual page for nslookup(1Mtcp). recurse Sets the query type to recursive. When toggled to norecurse, nslookup performs iterative queries. querytype=type Sets the query type to the DNS data type specified. Common types include a (Address), any (any data type), mx (Mail Exchanger), and ns (Name Server). retry=n Resends the query n times before giving up. root=root server Sets the root server to the server you enter. timeout=n The period of time nslookup waits for a response after the query is sent. This period doubles between each retry. You can save any of these options in a .nslookuprc file in your home directory. The format of this file, which is searched for each time you invoke nslookup, is one set command per line. Here is an example, which sets the query type to address records, the domain to mynet.com, and sets the timeout on requests to 10 seconds: set querytype=a set domain=mynet.com set timeout=10 Querying a single name or address To issue a simple query from the command line, use one of the following forms of the command: nslookup name nslookup IP_address nslookup should return the desired answer by querying the default server. To query a different server, enter one of the following forms of the command: nslookup name server nslookup IP_address server
-
LTM Migration compare OLD to NEW
When migrating you want to make sure you don't miss anything so here are a few commands that I run to help me make sure what was on the old is on the new. CONFIGURATION PHASE Virtual Servers First objective is to check to make sure all the Virtual Servers are present. If you aren't changing IP addresses then all I grab is the destination field since in many cases the name and/or partition may change. For example we are moving to deploying all our Virtual Servers using JSON format and the Partition is now the IP address so all pools, profiles etc that are specific to that virtual server would be found under that partition/ip address. Anyhow, so how do you get all the destination IPs from all partitions? Just run this command: tmsh -q -c 'cd / ; show ltm virtual recursive' | grep -i "destination " > /var/tmp/vs-destination-old_$HOSTNAME"."$(date +%Y%m%d).txt Why do I use show versus list? List command will show the destination IP with the port in the common name like 443 would be listed as https but on the show command its always the port number no matter how it was configured which is consistent which also makes it easier to do compares. Certificates and Keys AWAF Policies DEPLOYMENT PHASE - BEFORE MIGRATION DEPLOYMENT PHASE - AFTER MIGRATION VALIDATION PHASE
-
Bash Script to export VS-CPFL-CRT-CIPHERs
Here is a very helpful script that can be used to export Virtual Server Profile Certificate Ciphers I personally create a file called: show-vs-cpfl-cert-ciphers.sh Then I make it executable: chmod 755 show-vs-cpfl-cert-ciphers.sh Now copy the code below and paste it in the new file #!/bin/bash # Search /config and sub directories (partitions) for bigip.conf files LIST=`find /config -name bigip.conf | xargs awk '$2 == "virtual" {print $3}' 2> /dev/null | sort -u` echo "Virtual: Profile: Certificate: Ciphers:" echo "__________________________________________________________" for VAL in ${LIST} do PROF=`tmsh show /ltm virtual ${VAL} profiles 2> /dev/null | grep -B 1 " Ltm::ClientSSL Profile:" | cut -d: -f4 | grep -i "[a-z]" | sed s'/ //'g| sort -u` test -n "${PROF}" 2>&- && { VIRTS=`expr $VIRTS + 1` for PCRT in ${PROF} do CERT=`tmsh list /ltm profile client-ssl ${PCRT} | awk '$1 == "cert" {print $2}' 2> /dev/null | sort -u` test -n "${CERT}" 2>&- && { CIPHERS=`tmsh list /ltm profile client-ssl ${PCRT} ciphers | grep ciphers | awk '{print $2}'` echo "${VAL} ${PCRT} ${CERT} ${CIPHERS}" } done } done echo "Virtual server count: ${VIRTS}" Finally you can run the newly created file: ./show-vs-cpfl-cert-ciphers.sh My preference is to run it and capture the output to a txt file that I can manipulate later to sort by each of the fields so I run the command this way: ./show-vs-cpfl-cert-ciphers.sh > /var/tmp/how-vs-cpfl-cert-ciphers_output_$HOSTNAME"."$(date +%Y%m%d).txt Hope this helps you out as it has helped me
-
AWAF Policies
Trying to migrate an LTM from old school to doing to via JSON and this particular LTM has AWAF Resource Provisioned so its what makes it difficult AND we are running version 15.x which only supports exporting policies into XML. In newer versions (16.x and 17.x) the AWAF policies can be exported into JSON format. We exported the policies and imported them and now when you go to a child policy and you want to update it, the settings are grayed out like shown here But it use to look like this here where you can Add and do changes. It took a minute to understand what's happening but I eventually discovered that inheritance is taking place so you could make the changes at the parent policy but it would affect all child policies. If you want to edit just certain child policies then you will need to go to Inheritence Settings on the child policy and click Decline and you will see that button change from Decline to Declined. Below is a picture of what it looks like.. (in red is declining inheritance and in gray it is keeping the inheritance)
-
iQuery issues troubleshooting
REQUIREMENTS: For the BIG-IP DNS synchronization group members to properly synchronize their configuration settings, verify that the following requirements are in place: BIG-IP DNS synchronization group members must be running the same software version A BIG-IP DNS device should be running the same software version as other members in the synchronization group. BIG-IP DNS devices that are running different software versions will not be able to communicate and properly synchronize BIG-IP DNS configuration and zone files. For information about displaying the software version, refer to K8759: Displaying the BIG-IP software version. Synchronization parameters must be properly defined for all members Synchronization must be enabled and each device must have the same synchronization group name. You can define the synchronization parameters by navigating to: BIG-IP DNS 11.5.0 and later: DNS > Settings > GSLB > General BIG-IP GTM 10.0.0 through 11.4.1: System > Configuration > Device > GTM > General NTP must be configured on each device Before you can synchronize BIG-IP DNS systems, you must define the network time protocol (NTP) servers for all synchronization group members. Configuring NTP servers ensures that each BIG-IP DNS synchronization group member is referencing the same time when verifying the configuration data that needs to be synchronized. You can configure NTP by navigating to System > Configuration > Device > NTP. Port Lockdown must be set properly for the relevant self IP addresses Port lockdown is a security feature that specifies the protocols and services from which a self IP address can accept traffic. F5 recommends using the Allow Custom option for self IP addresses that are used for synchronization and other critical redundant pair intercommunications. You can configure port lockdown by navigating to Network > Self IPs. Note: Management-IP address are not compatible with iQuery; you should not use them as server IP addresses in the DNS server list. Configure the service ports shown in the following table for BIG-IP DNS operation on the specific self IP. Allowed Protocol Service Service Definition TCP 4353 iQuery TCP 22 SSH TCP 53 DNS UDP 53 DNS UDP 1026 Network Failover For further information on Port Lockdown behavior, please refer to K17333 listed in the Supplemental Information section below. TCP port 4353 must be allowed between BIG-IP GTM systems BIG-IP DNS synchronization group members use TCP port 4353 to communicate. You must verify that port 4353 is allowed between BIG-IP DNS systems. Compatible big3d versions must be installed on synchronization group members The big3d process runs on BIG-IP systems and collects performance information on behalf of the BIG-IP DNS system. For metrics collection to work properly, synchronization group members must run the same version of the big3d process. For more information about verifying big3d version information, refer to K13703: Overview of big3d version management. A valid device certificate must be installed on all members The device certificate is used by the F5 system to identify itself to a requesting F5 client system. The default device certificate, /config/httpd/conf/ssl.crt/server.crt, must be installed on each sync group member. You can verify the certificate validity by navigating to System> Device Certificates. EXPLANATION of DNS SYNC A BIG-IP DNS synchronization group is a collection of multiple BIG-IP DNS systems that share and synchronize configuration settings. You must meet several minimum requirements for BIG-IP DNS synchronization group members to communicate and synchronize properly. Starting in 11.x, the BIG-IP DNS system uses a commit_id structure, which is linked to an MCP transaction, as a timestamp when updating the configuration for a given sync group. The BIG-IP DNS sync group communication flow works as follows: The Configuration utility or the TMOS Shell (tmsh) communicates configuration changes to the mcpd process. The mcpd process forwards the new configuration in its entirety to the local gtmd process. The gtmd process updates the commit_id value and writes the new configuration to the /config/bigip_gtm.conf file. The local big3d process begins advertising the updated commit_id value using heartbeat messages transmitted to all remote gtmd processes. When a remote gtmd process notices that the peer BIG-IP DNS system has a newer commit_id value, the remote gtmd invokes the iqsyncer utility to pull the newer configuration. The iqsyncer utility connects to the big3d process of the BIG-IP DNS system with the newer commit_id and requests the changes between the newer commit_id and its current commit_id. The big3d process connects to its mcpd process and if the differences between commit_ids exist in the incremental config sync cache, then just these incremental changes are passed back. If not, the full configuration are passed in one or more messages. The big3d process then transmits those messages back to the requesting iqsyncer utility, and iqsyncer passes the new configuration directly to its own mcpd process, which loads it into memory. After the mcpd process receives the new configuration, it passes the configuration to its own gtmd process, which updates its timestamp with the commit_id of the source BIG-IP DNS system, and writes the configuration to the /config/bigip_gtm.conf file.
-
Unable to Redirect using Policy
Ran into an issue last night where I had to redirect https://example.thezah.com/ to https://example.thezah.com/?idp_id=two Attempted a few different way of redirecting the URI in the Policy and they all didn't work. Ran into a few issues... When creating the Redirect_URI policy under the do the following: Replace - HTTP URI - path with value /?idp_id=two at request time What would happen is when you enter the value /?idp_id=two and save F5 would change it to /\?idp_id=two and my assumption is its using reg-ex to escape the ? so I used the URL encode for question mark which is %3F so it looked like /%3Fidp_id=two and still no luck. Then realized they were breaking because to use policies you need an HTTP Profile (Client) of http. Then we added an SSL Profile (Client) with the FQDN in for example.thezah.com and also add the F5 default SSL Profile (Server) called serverssl The Policy still wouldn't work so created an iRule like the one below when HTTP_REQUEST { if { [HTTP::uri] equals "/" } { HTTP::uri "/?idp_id=two" } } Assigned the iRule to the Virtual Server Resources and now we are in business.
-
Using BIG-IQ to troubleshoot
If you have BIG-IQ in your environment to help manage/monitor your applications then let me help understand how to use some cool features of BIG-IQ. Many times you have several F5's in your environment and trying to identify what F5 has the application you need to troubleshoot is kind of a pain in the butt unless you have BIG-IQ. First thing I do is if someone says they have an issue with their application is I ask for the FQDN or the URL that is having issues. Next thing I do is go to BIG-IQ and click on the Configuration Tab then click on Virtual Servers and you get a screen like the one below You can enter the fqdn in the filter box on the right and if that doesn't work because the name of the virtual server may differ, go to your command prompt and do a dig on the FQDN to get the IP Address and come back to BIG-IQ and enter the IP Address in the Filter box on the right. What if you get nothing still? Then a few things could be happening Maybe there application doesn't go through the F5 BIG-IQ is only up to date if the application was deployed via JSON/AS3 and for those legacy apps that didn't use JSON/AS3 to deploy, under the Devices Tab you must click those legacy boxes and click import on the services to keep the BIG-IQ database up to date.
-
Hairband Party Songs
These are some of my favorites (in no order) Fast Songs 38 Special - Hold On Loosely AC/DC - Back in Black AC/DC - You Shook Me All Night Long AC/DC - Thunderstruck AC/DC - T.N.T. AC/DC - Highway to Hell Aerosmith - Dude (Looks Like A Lady) Aerosmith - Same Old Song and Dance Alice Cooper - Poison Alice Cooper - Is It My Body Anthrax - Caught In A Mosh [heavy] Anthrax - Among The Living Autograph - Turn up the Radio Autograph - Blondes In Black Cars Bad Company - Can't Get Enough Bad Company - Rock n Roll Fantasy Billy Squier - The Stroke Billy Squier - Everybody Wants You Billy Squier - Lonely Is The Night Billy Squier - My Kinda Lover Black Sabbath - Iron Man Black Sabbath - War Pigs Bon Jovi - Livin On A Prayer Bon Jovi - You Give Love A Bad Name Bon Jovi - Bad Medicine Bryan Adams - Summer of '69 Cinderella - Shake Me Cinderella - Gypsy Road Danzig - Mother Danzig - Twist of Cain David Lee Roth - Just Like Paradise Def Leppard - Pour Some Sugar On Me Def Leppard - Rock Of Ages Def Leepard - Photograph Deep Purple - Smoke On The Water Dio - Rainbow in the Dark Dire Straits - Money for Nothing Dokken - Breaking the Chains Faith No More - Epic Faith No More - The Real Thing Foreigner - Juke Box Hero Foreigner - Hot Blooded Grand Funk Railroad - We're An American Band Guns N Roses - Sweet Child O Mine Guns N Roses - Paradise City Guns N Roses - Welcome To The Jungle Heart - Barracuda Iron Maiden - The Trooper Iron Maiden - Run to the Hills Iron Maiden - 2 Minutes to Midnight Rick Springfield - Jessie's Girl Jane's Addiction - Jane Says Jane's Addiction - Mountain Song Joan Jett - I Love Rock N Roll Joan Jett - I Hate Myself for Loving You Journey - Any Way You Want It Journey - Separate Ways Judas Priest - Breaking the Law Judas Priest - You've Got Another Thing Coming Judas Priest - Living After Midnight Kansas - Carry on Wayward Son KISS - Rock and Roll all Nite KISS - Strutter KISS - I Love It Loud Krokus - Living Colour - Cult of Personality Loverboy - Working for the Weekend Megadeth - Symphony of Destruction Metallica - For Whom The Bells Tolls Metallica - One Metallica - Seek & Destroy Metallica - Master of Puppets Motley Crue - Girls, Girls, Girls Motley Crue - Dr. Feelgood Motley Crue - Kickstart My Heart Motley Crue - Shout At The Devil Motorhead - Ace of Spades Mountain - Mississippi Queen Night Ranger - Don't Tell Me You Love Me Ozzy Osbourne - Crazy Train Ozzy Osbourne - Bark at the Moon Pantera - Cowboys from Hell Pat Benatar - Hit Me With Your Best Shot Pat Benatar - Heartbreaker Poison - Nothin' But A Good Time Poison - Unskinny Bop Queen - I Want It All Queen - Another One Bites The Dust Quiet Riot - Cum on Feel the Noize Quiet Riot - Metal Health Rainbow - Since You Been Gone Ratt - Round and Round Ratt - Lay It Down Riot - Swords and Tequilla Robert Palmer - Addicted to Love Rush - The Spirit of Radio Rush - Tom Sawyer Sammy Hagar - Heavy Metal Saxon - Wheels of Steel Scorpions - Rock You Like A Hurricane Scorpions - No One Like You Skid Row - 18 and Life Slayer - Raining Blood [heavy] Stevie Nicks - Edge of Seventeen Styx - Renegade Tesla - Modern Day Cowboy The Rolling Stones - Start Me Up The Rolling Stones - Brown Sugar The Who - Pinball Wizard The Who - Baba O'Riley Thin Lizzy - The Boys Are Back In Town Thin Lizzy - Jailbreak Twisted Sister - We're Not Gonna Take It Twisted Sister - I Wanna Rock Twisted Sister - The Kids Are Back Van Halen - Panama Van Halen - Hot for Teacher Van Halen - Why Can't This Be Love Van Halen - Jump Van Halen - Runnin with the Devil Van Halen - Eruption Vixen - Edge Of A Broken Heart Volbeat - Lola Montez Warrant - Down Boys Warrant - Cherry Pie Whitesnake - Here I Go Again Whitesnake - Fool for Your Loving Winger - Seventeen XYZ - Inside Out ZZ Top - Can't Stop Rockin ZZ Top - Gimme All Your Lovin Slow Songs Cinderella - Don't Know What You Got (Till It's Gone) Bon Jovi - Wanted Dead Or Alive Poison - Every Rose Has Its Thorn Danzig - How the Gods Kill
-
Splunk Request Logging
SCENARIO Walking through troubleshooting since the virtual server will show up, the pool will show up but going to the URL the application doesn't come up as long as on the logging profile you have Respond on error enabled and your logging pool has no available members. BELOW IS AN EXAMPLE VIRTUAL SERVER ltm virtual /Integration/vs.sim1.102799.qa.enterpriseremarketing.int.thezah.com.443 { destination /Integration/10.46.65.206:443 ip-protocol tcp last-modified-time 2021-11-15:10:18:54 mask 255.255.255.255 partition Integration persist { cookie { default yes } } pool /Integration/pool.sim1.102799.qa.enterpriseremarketing.int.thezah.com.443 profiles { http { } logprofile { } oneconnect { } serverssl { context serverside } ssl.client.qa.enterpriseremarketing.int.thezah.com { context clientside } tcp-lan-optimized { } } rules { /Integration/irule.qa.enterpriseremarketing.int.thezah.com.content.redirect } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { pool MOD_SNAT_POOL type snat } translate-address enabled translate-port enabled vs-index 1492 } Run a tcpdump against the IP address of the Virtual Server to see what's going on since Virtual Server shows up, Pool shows up tcpdump -s0 -nni 0.0:nnnp -vvv -w /var/tmp/qa.enterprise_20200312.pcap host 10.46.65.206 In order to utilize the Wireshark F5 plugin, you need to flag the tcpdump command appropriately with -s0 and setting the level of noise by flagging the interface with a colon followed by a single, double, or triple n for, respectively, low, medium, and high details. In Wireshark use the display filter: f5ethtrailer.rstcausetxt in order to get the same screen as show above So from this you can see the F5 is RESETTING the connection (F5RST) and come to find out its because of the below setting in an attached logprofile Now the when a virtual server has the logprofile attached it sends an extreme amount of data to Splunk so it only gets turned on critical applications or if you are troubleshooting an application so you can get more log data to help find out whats going on exactly. With Respond on Error = Enabled it means when the Splunk Servers go down then the application also goes down. With Respond on Error = Disabled (which is the default setting) then when the logging server goes down, the applications will continue to function. Easy CLI command to change this to disable is tmsh modify ltm profile request-log logprofile proxy-respond-on-logging-error no Hope you find this helpful. Also note that the wireshark capture above to get the F5RST to be displayed you have to either install the wireshark plugin on Wireshark 2.5 and older and on Wireshark 2.6 and newer you just need to enable it via Analyze - Enabled Protocols - F5 Ethernet trailer - f5ethtrailer Now you can
-
MAF User Management
There may be times when a user is asked to “Reauthenticate” when they are logging into the child sites. why does this happen? usually it’s because the passwords are not in sync meaning the password was changed on the child site somehow instead of the main/master site (eventguyz.com) how to fix? Have an administrator update ur password on both master site and child site. You can then change/update your password by going to the main/master website (https://eventguyz.com), log in and in the top right corner click on your name and click Account Settings. There you will see the ability to update your password. Once completed you can now log into any of the child sites with the updated password. For administrators it’s a tad different. You need to log into each the backend (or have another administrator do it) change ur password.
-
Davison Robotics
Today is the last day supporting the website. Some bad eggs. No longer supporting the following websites davisonrobotics.com davisonrobotics.net davisonrobotics.org team10058.com team9514.com team3534.com DONE! Yes this logo is something provided by EventGuyZ
-
Davison Robotics
Davison Robotics is part of an extra curricular activity offered to students of the Davison Community Schools that offers the ability to learn STEM (Science, Technology, Engineering, and Mathematics) and is part of FIRST. Current Davison Community School FIRST Teams include: Davison High School (Team 3534) Davison Middle School (Team 10058, Team 9514 ) Davison Intermediate School ( ) Davison Elementary School ( ) EventGuyZ became a supporter and addressed the lack of a davisonrobotics.com website in early 2018. They were using team3534.com but nothing to represent any of the other Davison Robotics teams so we registered davisonrobotics.com and created a general Davison Robotics logo Then we created email accounts for the domain @davisonrobotics.com since the turnover on volunteers and board members is pretty often so when volunteers contact supporters and/or vendors the email will be saved for the next person. All we do is setup a forward to personal email of volunteer so they can see the email come into there mailbox (still leaving the original in the mailbox on the server). To respond, they can email with [email protected] so everything is saved. All working beautifully. Initial website used Wordpress which was slow... very very slow. Was getting attacked on a regular basis (unsuccessfully) but the investment to build the website on Wordpress and add all the requested features was taking more money than we felt comfortable donating so we had to regroup and strategize on a better solution. We ended up building an Invision Community website that was beautiful, functional and half the cost. Also provided much better security. Kids and Coaches were using the site successfully and then began the downward spiral which is why the website no longer exists. We did this to help the kids and support volunteers since they are giving up their time to help the students be successful but being ungrateful for the thousands invested breaks that relationship very easily. We wish Davison Robotics much success. You can follow them on Facebook.
-
F5 GTM-DNS Sync Group
This is to help better explain the purpose of a sync group on the F5 GTM's or otherwise known as BIG-IP DNS. The following figure shows that, after a configuration change is made on the Los Angeles BIG-IP DNS system, the local big3d process initiates an iQuery connection to BIG-IP DNS sync group members in New York and Europe and advertises the updated configuration to the remote gtmd processes. Synchronization details When you configure BIG-IP DNS synchronization, the sync group members share and synchronize BIG-IP DNS configuration objects and metrics data. The following table lists the relevant configuration objects and whether the objects are synchronized. BIG-IP DNS configuration object Synchronized Wide IP addresses Yes Data centers Yes Servers Yes Virtual servers Yes Links Yes GSLB iRules Yes Topology records / Regions Yes Distributed applications Yes GSLB global settings Yes GSLB monitors Yes DNSSEC zones / Keys Yes DNS zone files Not synchronized by default Named configuration Not synchronized by default Listener addresses No DNS express zones No DNS cache No Synchronization group requirements Before you configure synchronization, you should be aware of the requirements for BIG-IP DNS synchronization group members to communicate and synchronize properly which are found on F5 K13734, but the high level summary of it is this for the BIG-IP DNS synchronization group members to properly synchronize their configuration settings. Verify that the following requirements are in place: BIG-IP DNS synchronization group members must be running the same software version A BIG-IP DNS device should be running the same software version as other members in the synchronization group. BIG-IP DNS devices that are running different software versions will not be able to communicate and properly synchronize BIG-IP DNS configuration and zone files. For information about displaying the software version, refer to K8759: Displaying the BIG-IP software version. Synchronization parameters must be properly defined for all members Synchronization must be enabled and each device must have the same synchronization group name. You can define the synchronization parameters by navigating to on BIG-IP DNS 11.5.0 and later: DNS > Settings > GSLB > General NTP must be configured on each device Before you can synchronize BIG-IP DNS systems, you must define the network time protocol (NTP) servers for all synchronization group members. Configuring NTP servers ensures that each BIG-IP DNS synchronization group member is referencing the same time when verifying the configuration data that needs to be synchronized. You can configure NTP by navigating to System > Configuration > Device > NTP. Port Lockdown must be set properly for the relevant self IP addresses Port lockdown is a security feature that specifies the protocols and services from which a self IP address can accept traffic. F5 recommends using the Allow Custom option for self IP addresses that are used for synchronization and other critical redundant pair intercommunications. You can configure port lockdown by navigating to Network > Self IPs. Note: Management-IP address are not compatible with iQuery; you should not use them as server IP addresses in the DNS server list. Configure the service ports shown in the following table for BIG-IP DNS operation on the specific self IP. Allowed Protocol Service Service Definition TCP 4353 iQuery TCP 22 SSH TCP 53 DNS UDP 53 DNS UDP 1026 Network Failover For further information on Port Lockdown behavior, please refer to K17333 listed in the Supplemental Information section below. TCP port 4353 must be allowed between BIG-IP GTM systems BIG-IP DNS synchronization group members use TCP port 4353 to communicate. You must verify that port 4353 is allowed between BIG-IP DNS systems. Compatible big3d versions must be installed on synchronization group members The big3d process runs on BIG-IP systems and collects performance information on behalf of the BIG-IP DNS system. For metrics collection to work properly, synchronization group members must run the same version of the big3d process. For more information about verifying big3d version information, refer to K13703: Overview of big3d version management. A valid device certificate must be installed on all members The device certificate is used by the F5 system to identify itself to a requesting F5 client system. The default device certificate, /config/httpd/conf/ssl.crt/server.crt, must be installed on each sync group member. You can verify the certificate validity by navigating to System > Device Certificates. Configuration review via GUI Enable synchronization on the system to ensure that the BIG-IP DNS system that is already installed on your network can share configuration changes with other BIG-IP DNS systems that you add to the BIG-IP DNS synchronization group. On the Main tab, click DNS > Settings > GSLB > General . The General configuration screen opens. Select the Synchronize check box. In the Group Name field, type the name of the synchronization group to which you want this system to belong. In the Time Tolerance field, type the maximum number of seconds allowed between the time settings on this system and the other systems in the synchronization group.The lower the value, the more often this system makes a log entry indicating that there is a difference. Tip: If you are using NTP, leave this setting at the default value of 10. In the event that NTP fails, the system uses the time_tolerance variable to maintain synchronization. Click Update. When a change is made on one BIG-IP DNS system in the BIG-IP DNS synchronization group, that change is automatically synchronized to the other systems in the group. Creating a data center on the existing BIG-IP DNS Create a data center on the existing DNS system to represent the location where the new BIG-IP DNS system resides. On the Main tab, click DNS > GSLB > Data Centers . The Data Center List screen opens. Click Create. The New Data Center screen opens. In the Name field, type a name to identify the data center. Important: The data center name is limited to 63 characters. In the Location field, type the geographic location of the data center. In the Contact field, type the name of either the administrator or the department that manages the data center. From the Prober Preference list, select the preferred type of prober(s). Option Description Inside Data Center By default, select probers inside the data center. Outside Data Center Select probers outside the data center. Specific Prober Pool Select one of the Probers from the drop-down list. When you want to assign a Prober pool at the data center level. Note: Prober pools are not used by the bigip monitor. From the Prober Fallback list, select the type of prober(s) to use if insufficient numbers of the preferred type are available. Option Description Any Available By default, select any available prober. Inside Data Center Select probers inside the data center. Outside Data Center Select probers outside the data center. None No fallback probers are selected. Prober fallback is disabled. Specific Prober Pool Select one of the Probers from the drop-down list. When you want to assign a Prober pool at the data center level. From the State list, select Enabled or Disabled. The default is Enabled, which specifies that the data center and its resources are available for load balancing. Click Finished. Defining a server on the existing BIG-IP DNS You must ensure that a data center where the new DNS system resides is available in the configuration of the existing BIG-IP® DNS before you start this task. You define a new server, on the existing BIG-IP DNS system, to represent the new BIG-IP DNS system. On the Main tab, click DNS > GSLB > Servers . The Server List screen opens. Click Create. The New Server screen opens. In the Name field, type a name for the server. Important: Server names are limited to 63 characters. From the Product list, select BIG-IP System. From the Data Center list, select the data center where the server resides. From the Prober Preference list, select the preferred type of prober(s). Option Description Inherit From Data Center By default, a server inherits the prober preference selection assigned to the data center in which the server resides. Inside Data Center A server selects the probers from inside the data center where the server resides. Outside Data Center A server selects the probers from outside the data center where the server resides. Specific Prober Pool Select one of the Prober pools from the drop-down list. When assigning the Prober pool at the server level. Note: Prober pools are not used by the bigip monitor. From the Prober Fallback list, select the type of prober(s) to be used if insufficient numbers of the preferred type are available. Option Description Inherit From Data Center By default, a server inherits the prober fallback selection assigned to the data center in which the server resides. Any Available For selecting any available prober. Inside Data Center A server selects probers from inside the data center where the server resides. Outside Data Center A server selects probers from outside the data center where the server resides. None No fallback probers are selected. Prober fallback is disabled. Specific Prober Pool Select one of the Probers from the drop-down list. When you want to assign a Prober pool at the server level. From the State list, select Enabled. In the BIG-IP System Devices area, click Add to add a device (server). Type a name in the Device Name field. Type an external (public) non-floating IP address in the Address field. If you use NAT, type an internal (private) IP address in the Translation field, and then click Add. Click Add. Click OK. From the Configuration list, select Advanced. Additional controls display on the screen. In the Health Monitors area, assign the bigip monitor to the server by moving it from the Available list to the Selected list. From the Availability Requirements list, select one of the following and enter any required values. Option Description All Health Monitors By default, specifies that all of the selected health monitors must be successful before the server is considered up (available). At Least The minimum number of selected health monitors that must be successful before the server is considered up. Require The minimum number of successful probes required from the total number of probers requested. From the Virtual Server Discovery list, select how you want virtual servers to be added to the system. Option Description Disabled The system does not use the discovery feature to automatically add virtual servers. This is the default value. Use this option for a standalone BIG-IP DNS system or for a BIG-IP DNS/LTM® combo system when you plan to manually add virtual servers to the system, or if your network uses multiple route domains. Enabled The system uses the discovery feature to automatically add and delete virtual servers. Use this option for a BIG-IP DNS/LTM combo system when you want the BIG-IP DNS system to discover LTM virtual servers. Enabled (No Delete) The system uses the discovery feature to automatically add virtual servers and does not delete any virtual servers that already exist in the configuration. Use this option for a BIG-IP DNS/LTM combo system when you want the BIG-IP DNS system to discover LTM virtual servers. In the Virtual Server List area, if you selected Disabled from the Virtual Server Discovery list, specify the virtual servers that are resources on this server. In the Name field, type the name of the virtual server. In the Address field, type the IP address of the virtual server. From the Service Port list, select the port the server uses. Click Add. Click Finished. Note: The gtmd process on each BIG-IP DNS system will attempt to establish an iQuery® connection over port 4353 with each self IP address defined on each server in the BIG-IP DNS configuration of type BIG-IP. Allow port 4353 in your port lockdown settings for iQuery® to work. The Server List screen opens displaying the new server in the list. The status of the newly defined BIG-IP DNS system is Unknown, because you have not yet run the gtm_add script. Running the gtm_add script Before you start this task, you must determine the self IP address of a DNS system in the BIG-IP® DNS synchronization group to which you want to add another BIG-IP DNS. You run the gtm_add script on the BIG-IP DNS system you are adding to your network to acquire the configuration settings from a BIG-IP DNS system that is already installed on your network. For additional information about running the script, see SOL13312 on AskF5.com (www.askf5.com). Note: The BIG-IP DNS and other BIG-IP systems must have TCP port 22 open between the systems for the script to work. You must perform this task from the command-line interface. Log in as root to the BIG-IP DNS system you are adding to your network. Run this command to access tmsh. tmsh Run this command to run the gtm_add script run gtm gtm_add Press the y key to start the gtm_add script. Type the IP address of the BIG-IP DNS system in the synchronization group to which you are adding this BIG-IP DNS system. Press Enter. If prompted, type the root password. Press Enter. The BIG-IP DNS system you are installing on your network acquires the configuration of the BIG-IP DNS system already installed on your network. Implementation result The new BIG-IP® DNS system that you added to the network is a part of a BIG-IP DNS synchronization group. Changes you make to any system in the BIG-IP DNS synchronization group are automatically propagated to all other BIG-IP DNS systems in the group. Troubleshooting BIG-IP DNS sync connections (11.x - 16.x) tmsh The tmsh utility lists failing server objects as Offline and a failing iQuery connection as Not Connected. The following table lists tmsh commands that you can use to check the status of BIG-IP DNS synchronization group members and iQuery connections. tmsh component Description Example commands server Summary of defined DNS/GTM server objects tmsh list /gtm server all tmsh show /gtm server all iquery Summary of iQuery statistics tmsh show /gtm iquery all gtm Summary of DNS/GTM statistics tmsh show /gtm Note: All members that participate in the iQuery mesh must be listed in the Server List. If a member of the iQuery mesh is not included in the Server List, it may result in some or all monitors intermittently or consistently failing. The monitors fail any time the big3d agent on the missing member (server) is expected to perform and report the monitor status. This can result in virtual servers being marked offline with a reason of no reply from big3d: timed out. Verify required configuration elements for synchronization group members For BIG-IP DNS synchronization group members to communicate and synchronize properly, you must verify that certain requirements are in place. To do so, review the following checklist. Sync requirement Description Configuration utility location tmsh Software versions Run the same software version for synchronization group members System > Software Management tmsh show /sys software Sync settings Use the same synchronization group settings for all members DNS > Settings > GSLB > General (BIG-IP 11.5.0 and later) System > Configuration > Global Traffic > General (BIG-IP 11.4.1 and earlier) tmsh list /gtm global-settings general all-properties NTP Configure NTP for all members System > Configuration > Device > NTP tmsh list /sys ntp servers Port Lockdown Use the Allow Default option for self IPs that process iQuery traffic Network > Self IPs tmsh list /net self allow-service iQuery port Verify that TCP port 4353 is allowed on interconnecting devices Not Applicable Not Applicable big3d versions Run the same big3d version on all members. Note: The big3d version should not be older than the host BIG-IP version. Not Applicable big3d -v /shared/bin/big3d -v Review log files Reviewing the log files is one way to determine the cause of synchronization/iQuery connection issues. The system logs global traffic events to the /var/log/gtm file. Some of the logging related to synchronization/iQuery connection issues is as follows: Device certificate messages The BIG-IP system uses SSL certificates for inter-device communication using the iQuery protocol. If device certificates are missing, expired, or contain duplicate common name (CN) entries with certificates on one of the synchronization group members, the system is marked Offline and logs an error message to the /var/log/gtm file that appears similar to the following example: SSL error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed When creating or renewing BIG-IP device certificates, use the following guidelines: Device certificates should have unique and meaningful subject data. For example, the CN field should match the hostname of the BIG-IP system in which the certificate was created. When possible, create device certificates with an extended expiration date. Make sure that SSL certificates are not expired. iQuery Connectivity messages The iQuery protocol uses TCP port 4353 to connect to synchronization group members. The system logs a successful iQuery connection to the /var/log/gtm file. For example: gtmd[8472]: 011ae020:5: Connection in progress to <iquery_peer> gtmd[8472]: 011ae01c:5: Connection complete to <iquery_peer>. Starting SSL handshake gtmd[11895]: 011a5003:1: SNMP_TRAP: Server /Common/<hostname> (ip=<iquery_peer>) state change red --> green gtmd[11895]: 011a5008:1: SNMP_TRAP: BIG-IP GTM /Common/<hostname> (<iquery_peer>) joined sync group default If the iQuery protocol is blocked; for example, by a router ACL, or packet filter, the BIG-IP DNS system marks its iQuery peer as Unavailable and attempts to reestablish the iQuery connection every 10 seconds. When this behavior occurs, a log sequence appears in the /var/log/gtm file that appears similar to the following example: gtmd[11895]: 011a500c:1: SNMP_TRAP: Box <iquery_peer> state change green --> red (Box <iquery_peer> on Unavailable) gtmd[11895]: 011a5004:1: SNMP_TRAP: Server /Common/<hostname> (ip=<iquery_peer>) state change green --> red (No communication) gtmd[8472]: 011ae020:5: Connection in progress to <iquery_peer> gtmd[8472]: 011ae020:5: Connection in progress to <iquery_peer> gtmd[8472]: 011ae020:5: Connection in progress to <iquery_peer> gtmd[8472]: 011ae020:5: Connection in progress to <iquery_peer> NTP messages The Synchronization Time Tolerance setting specifies the number of seconds that one system clock can be out of sync with another system clock in the synchronization group. If the time difference between synchronization group members is greater than the Synchronization Time Tolerance value, the system logs a message to the /var/log/gtm file that appears similar to the following example: gtmd[11895]: 011a0022:2: Time difference between GTM /Common/B3900-242 and me is 486 seconds -- Make sure NTP is running and GTM times are in sync This error message is an indication that NTP may not be configured on one or more synchronization group members. Troubleshoot iQuery connectivity BIG-IP DNS systems in a synchronization group create an iQuery mesh across synchronization group members. For example, the local BIG-IP DNS system's gtmd process opens an iQuery connection to its own big3d process, and to remote synchronization group member's big3d process. There may be occasions when you must test iQuery connectivity between synchronization group members. For example, if log messages indicate that a BIG-IP DNS system has marked its iQuery peer as Unavailable, you can perform the following troubleshooting procedure to test TCP port 4353 connectivity: Impact of procedure: Performing the following procedure should not have a negative impact on your system. Log in to the command line. To verify the iQuery connection status, enter the following netstat command: netstat -na |grep 4353 The following netstat output indicates that the local system (10.11.16.238) is listening on port 4353 and has an iQuery connection established to its own big3d process. In addition, the local system and its iQuery peer (10.11.16.242) have established an iQuery mesh: Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 :::4353 :::* LISTEN tcp 0 0 ::ffff:10.11.16.238:52794 ::ffff:10.11.16.238:4353 ESTABLISHED tcp 0 0 ::ffff:10.11.16.238:4353 ::ffff:10.11.16.242:58779 ESTABLISHED tcp 0 0 ::ffff:10.11.16.238:4353 ::ffff:10.11.16.238:52794 ESTABLISHED tcp 0 0 ::ffff:10.11.16.238:46882 ::ffff:10.11.16.242:4353 ESTABLISHED If the synchronization group iQuery mesh is incomplete, you can use the iqdump command to determine if the iQuery packets arrive at the destination. If the iQuery channel is not established, iqdump returns with an SSL error similar to the following example: iqdump 10.10.10.20 46947856243768:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:s3_clnt.c:1168: Note: If the iqdump command returns a connection refused message, you should ensure connectivity for the iQuery channel is allowed, such as ensuring port 4353 is allowed by the self IP addresses on each system and devices in between. You may need to restart the big3d process to recover from the connection-refused condition. If the iQuery channel is established, iqdump returns XML similar to the following example: iqdump 10.10.10.20 <!-- Local hostname: lc1.example.com --> <!-- Connected to big3d at: ::ffff:10.10.10.10:4353 --> <!-- Subscribing to syncgroup: default --> <!-- Tue May 6 09:55:43 2014 --> <xml_connection> <version>11.5.1</version> <big3d>big3d Version 11.5.1.0.0.110</big3d> Verify device SSL certificates Each synchronizing group member must have a valid SSL device certificate installed in the /config/httpd/conf/ssl.crt/ directory for iQuery connections to succeed. If log messages indicate an issue with a device certificate on one of the synchronization group members, you can verify the certificate status by performing the following procedure: Note: SSL certificates signed by a third-party certificate authority (CA) must include both the client authentication (clientAuth) and server authentication (serverAuth) extended key usage (EKU) extensions, to allow use by both server and client applications. For example, the big3d process operates as a server (serverAuth), while the gtmd process operates as a client (clientAuth). For information, refer to K7717: BIG-IP DNS and Link Controller support for third-party SSL certificates. Impact of procedure: Performing the following procedure should not have a negative impact on your system. Log in to the command line. Check the status of the device certificate by entering the following command: openssl x509 -noout -text -in /config/httpd/conf/ssl.crt/server.crt Verify the certificate validity date and confirm whether the certificate is expired. If necessary, renew the certificate. To do so, refer to K16951115: Changing the BIG-IP DNS system device certificate using the Configuration utility. Troubleshoot daemons Impact of procedure: Performing the following procedure should not have a negative impact on your system. The tmm, mcpd, big3d, and gtmd processes are all critical to synchronizing BIG-IP DNS configurations. To confirm that the daemons are running as expected, use the tmsh command. For example, to confirm the status of the tmm, mcpd, big3d, and gtmd processes, enter the following command: tmsh show sys service tmm mcpd big3d gtmd If the mcpd process is consuming more than 90 percent of a CPU, and synchronizing actions, such as saving the configuration, may fail. To check the CPU usage for the mcpd process, enter the following command: top -p `pidof mcpd` To quit, enter q. Troubleshoot synchronization group members using the server type Starting from BIG-IP 12.x, you can use the Server Type field from the tmsh show /gtm iquery command output to determine if the listed BIG-IP DNS devices are fully setup to be in the same BIG-IP DNS synchronization group. If the BIG-IP DNS device is fully setup to be in the same BIG-IP DNS synchronization group as the remaining listed BIG-IP DNS devices, the command output would have the value of BIGIP-DNS for Server Type as shown in the following example: ----------------------------------------------------------- Gtm::IQuery: 192.168.74.129 ----------------------------------------------------------- Server b100 Server Type BIGIP-DNS Note: The previous example output is truncated for brevity. If the BIG-IP DNS device is not properly setup to be in the same BIG-IP DNS synchronization group as the remaining listed BIG-IP DNS devices, the command output would have the value of BIGIP for Server Type as shown in the following example: Important: For remote BIG-IP LTM devices that are integrated into the network with BIG-IP DNS, their Server Type continues to indicate as BIGIP. -------------------------------------------------- Gtm::IQuery: 192.168.74.130 -------------------------------------------------- Server b101 Server Type BIGIP Note: The previous example output is truncated for brevity. In the case of the BIG-IP DNS device not properly setup, you may want to re-run the gtm_add utility on the affected BIG-IP DNS device again.
-
Splunk Request Logging
Assumptions: Log volume will be huge and will only turn for critical applications that too in production. This can be tested in Dev/Pre-prod prior moving to the production but need to be turned off immediately.This will not cause performance issues because of High-speed logging HSL feature. This logging feature can also be turned on for troubleshooting purposes if required. Dependencies: Enterprise splunk team should provision dedicated storage for the new applications with F5 logging feature turned on a permanent basis. Estimation: If we turn on logging for all applications in production, current rough estimate would be 1TB logs per day. Introduction: Request logging feature is inline F5 feature that replace the functionality of the splunk logging irule. This request logging feature provide metrics such as client-ip, elapsed time, request type & details, response code, pool member etc.These logs integrated to splunk will provide lot of details about a particular application behind a single vip. Alerts, dashboard and reports can also be generated based of the metrics available per application basis. These logs will provide lot of help in troubleshooting and understanding more about the application for sustain teams. Sample Log: Description of log fields:- Time stamp of the log in splunk 3/4/2019 15:57 Device ip sending log 10.47.194.101 descriptive field for splunk queries Splunk Logging Device_hostname txsat1slbdv03.thezah.corp Requesttime_milli-sec 1551736658724 Requesttime_micro-sec 1551736658724868 Client_ip 10.7.156.86 Client_port 63710 vip_name /Development/vs.dev.103867.aap-api-cit2.thezah.corp vip_ip 10.47.37.242 vip_port 443 pool_name /Development/pool.dev.103867.aap-api-cit2.thezah.corp" snat_ip 10.47.34.35 snat_ip_port 46235 poolmember_ip 10.47.49.66 poolmember_port 12000 http_method GET http_uri /sa-health/f5chk.html http_version HTTP/1.1 referrer " " user-agent Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36 http_statuscode 304 Responsetime_milli-sec 2 response_size 0 Responsetime_micro-sec 2056 Procedure:- Navigate to the following. Local Traffic -> Profiles -> Other -> Request Logging -> Create as per below fields. Template contents from image: "Splunk Logging"|"$BIGIP_HOSTNAME"|"$TIME_MSECS"|"$TIME_USECS"|"$CLIENT_IP"|"$CLIENT_PORT"|"$VIRTUAL_NAME"|"$VIRTUAL_IP"|"$VIRTUAL_PORT"|"$VIRTUAL_POOL_NAME"|"$SNAT_IP"|"$SNAT_PORT"|"$SERVER_IP"|"$SERVER_PORT"|"$HTTP_METHOD"|"$HTTP_URI"|"$HTTP_VERSION"|"$Referer"|"${User-agent}"|"$HTTP_STATCODE"|"$RESPONSE_MSECS"|$RESPONSE_SIZE|"$RESPONSE_USECS" Once logfile is created, apply to virtual server that need this logging feature turned on as per below command. #tmsh modify ltm virtual /Development/vs.dev.103867.aap-api-cit2.thezah.corp profiles add { logprofile } Note:- turning logprofile on would not cause any performance issues because of HSL logging feature used, it will only need additional storage assigned from splunk team to get the logs indexed in splunk. Verification:- Check logs in F5 index ( index=infra_network ) to make sure transaction logs coming in. index=infra_network “Splunk Logging” “aap-api-cit2.thezah.corp” latest=+15m
-
Troubleshooting proxy and/or 3rd party URLs
This is extremely useful troubleshooting external URLs going through proxy / eGTMs / iGTMs and all other sort of combinations. What i really like about it is it really gives good data for different touch points. 1.time_namelookup 2.time_connect 3.time_appconnect 4.time_pretransfer 5.time_redirect 6.time_starttransfer It helped me handling extrenal 3rd party URLs and their response times, handshake failures. ***************************************************************************************************************************************************** $ cat curl-format.txt time_namelookup: %{time_namelookup}\n time_connect: %{time_connect}\n time_appconnect: %{time_appconnect}\n time_pretransfer: %{time_pretransfer}\n time_redirect: %{time_redirect}\n time_starttransfer: %{time_starttransfer}\n ----------\n time_total: %{time_total}\n $ curl -p 10.43.196.140:80 -w "@curl-format.txt" -o /dev/null -k https://na60.mywiseguys/services/Soap/class/AllySFBridgeIntegrationService?wsdl % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 677 100 677 0 0 41924 0 --:--:-- --:--:-- --:--:-- 42312 time_namelookup: 0.000 time_connect: 0.007 time_appconnect: 0.000 time_pretransfer: 0.008 time_redirect: 0.000 time_starttransfer: 0.016 ---------- time_total: 0.016 <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 405 Only POST allowed</title> </head> <body><h2>HTTP ERROR 405</h2> <p>Problem accessing /services/Soap/class/EventGuyZSFBridgeIntegrationService. Reason: <pre> Only POST allowed</pre></p><hr /><br/> <!-- Body events --> <script type="text/javascript">function bodyOnLoad(){if(window.PreferenceBits){window.PreferenceBits.prototype.csrfToken="null";};}function bodyOnBeforeUnload(){}function bodyOnFocus(){}function bodyOnUnload(){}</script> </body> </html> <!-- ................................................................................................... ................................................................................................... ................................................................................................... ................................................................................................... -->time_namelookup: 0.012 time_connect: 0.013 time_appconnect: 0.251 time_pretransfer: 0.251 time_redirect: 0.000 time_starttransfer: 0.507 ---------- time_total: 0.507 ***************************************************************************************************************************************************** Here is what each value represents, according to curl’s manual page: lookup: The time, in seconds, it took from the start until the name resolving was completed. connect: The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed. appconnect: The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0) pretransfer: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved. redirect: The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3) starttransfer: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result. total: The total time, in seconds, that the full operation lasted. The time will be displayed with millisecond resolution. Note:- Proxy details are below, replace proxies as & when requested in the curl command. User Proxies (from branches) DFW: 10.43.196.141,142,143,144,148,149 SAT: 10.47.196.141,142,143,144,148 System Proxies (from data center, system to system) DFW: 10.43.196.139, 140 SAT: 10.47.196.139, 140 NOTE: You must create the curl-format.txt file where it is that you are going to run the command. The contents need to be time_namelookup: %{time_namelookup}\n time_connect: %{time_connect}\n time_appconnect: %{time_appconnect}\n time_pretransfer: %{time_pretransfer}\n time_redirect: %{time_redirect}\n time_starttransfer: %{time_starttransfer}\n ----------\n time_total: %{time_total}\n SYNTAX of the command is curl -p <enter proxy IP:port> -w "@curlformat.txt" -o /dev/null -k <enter URL> BELOW you will find some examples of running the command Here is another example: Here is a FAILED request linux001:~ iSupportU$ curl -p 10.43.196.140:80 -w "@curl-format.txt" -o /dev/null -k https://na60.mywiseguys.com/services/Soap/class/EventGuyZSFBridgeIntegrationService?wsdl % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 799 100 799 0 0 9384 0 --:--:-- --:--:-- --:--:-- 9400 time_namelookup: 0.005050 time_connect: 0.043789 time_appconnect: 0.000000 time_pretransfer: 0.043842 time_redirect: 0.000000 time_starttransfer: 0.085053 ---------- time_total: 0.085140 time_namelookup: 0.066539 time_connect: 0.000000 time_appconnect: 0.000000 time_pretransfer: 0.000000 time_redirect: 0.000000 time_starttransfer: 0.000000 ---------- time_total: 225.609011 curl: (7) Failed to connect to na60.mywiseguys.com port 443: Operation timed out Here is a SUCCESSFUL request (reason this is being truncated is because the entire page is displayed in html text format and its just too much to copy and paste) linux001:~ iSupportU$ curl -p 10.43.196.140:80 -w "@curl-format.txt" -o /dev/null -k https://confluence.int.eventguyz.com % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 799 100 799 0 0 9385 0 --:--:-- --:--:-- --:--:-- 9400 time_namelookup: 0.005206 time_connect: 0.044339 time_appconnect: 0.000000 time_pretransfer: 0.044383 time_redirect: 0.000000 time_starttransfer: 0.085003 ---------- time_total: 0.085130 <!DOCTYPE html> <html> <head> <title>Dashboard - EventGuyZ Confluence Powered Wiki</title> <------- tons of stuff that I'm removing and skipping to the end ------→ <script type="text/javascript"> AJS.BigPipe = AJS.BigPipe || {}; AJS.BigPipe.metrics = AJS.BigPipe.metrics || {}; AJS.BigPipe.metrics.pageEnd = typeof window.performance !== "undefined" && typeof window.performance.now === "function" ? Math.ceil(window.performance.now()) : 0; AJS.BigPipe.metrics.isBigPipeEnabled = 'false' === 'true'; </script> </body> </html> time_namelookup: 0.004505 time_connect: 0.040007 time_appconnect: 0.184635 time_pretransfer: 0.184683 time_redirect: 0.000000 time_starttransfer: 0.256104 ---------- time_total: 0.272074 With an example of both you can now do comparisons and see the difference of a successful and failed connection attempt