The Challenges of Network Security

University Degree Mathematical and Computer Sciences

The Challenges of Network Security Remediation at a Regional University

A thesis

presented to

the faculty of the Department of Computer and Information Sciences

East Tennessee State University

In partial fulfillment

of the requirements for the degree

Master of Science in Computer Science

William R. Simons

May 2005

Dr. Qing Yuan, Chair

Dr. Phillip Pfeiffer

Mr. Steven Jenkins

Keywords: computer, system security, network security, security audit, security

hardening, vulnerability, remediation, Nessus, Nmap

ABSTRACT

The Challenges of Network Security Remediation at a Regional University

William R. Simons

This thesis describes challenges encountered during a year-long effort to

improve the security of the 3,300 node administrative computer network at East

Tennessee State University. The key remediation strategies used included

employing the vulnerability scanner Nessus to profile the network, analyzing the

scan results, and attempting to remove the most critical vulnerabilities found.

The project succeeded in decreasing known “high” criticality vulnerabilities on

campus by 26.1%, and confirmed four standard observations about the

challenges of network administration:

• Vulnerability scanning is a lengthy task best performed in parallel

and supported by automated data analysis.

• Securing a network is like trying to hit a moving target, due to an

ever-increasing proliferation of networked hosts, services enabled

by default install and lists of vulnerabilities to address.

• Failures of common sense are still among the primary threats to

network security.

• Failing to retain management support for the security hardening

process can jeopardize the project.

DEDICATION

This thesis would be incomplete without mention of the support given me by the

following people.

To,

Theresa and Rick. You had the maturity to understand what you could and

could not do in this world. You gave me a chance, by letting me go.

Mom and Dad. You took a four pound “preemie” and embraced him with a

lifetime of unconditional love. For eighteen years you attended scout meetings,

parent/teacher conferences, scholars’ bowls, baseball, football, and basketball

games, and preached the value of education. You raised me, and I credit you

for who I have become. Truly, God reserves a special place for altruistic people

like you.

Catherine Simons. You are my high school sweetheart and wife. You saw not

the boy I was, but the man I would become. You willingly but wrongly

accepted second place for too long. This thesis, of whatever value and quality,

is a small substitute.

Finally, Rick Simons. For nine years you selfishly put relationships and life on hold,

in pursuit of this end. You worked a full time job and endured countless

assignments, exams and their associated sleepless nights of preparation. You

did it, but never underestimate, underappreciate, nor forget the support,

understanding, and tolerance afforded you by your instructors, friends, and

family.

TABLE OF CONTENTS

Page

ABSTRACT ................................................................................................................................ 2

DEDICATION............................................................................................................................ 3

TABLE OF CONTENTS ................................................................................................................ 4

LIST OF FIGURES ........................................................................................................................ 6

Chapter

1. INTRODUCTION................................................................................................................... 7

1.1 CHALLENGES ENCOUNTERED............................................................................................... 8

2. BACKGROUND .................................................................................................................. 12

2.1 IDEA ORIGINS .................................................................................................................. 12

2.2 COMPUTER AND NETWORK SECURITY BACKGROUND............................................................ 13

2.3 THREATS AND FIXES TO NETWORK AND SYSTEM SECURITY........................................................ 14

2.3.1 Vulnerability Definition..................................................................................................14

2.3.2 Specific Threats to Network Security .........................................................................15

2.3.2.1 Cleartext Network Transmission.. .................................................................. 15

2.3.2.2 Misconfigured and/or Unpatched Applications.. ...................................... 16

2.3.2.3 Well-known or Default Install Accounts.. ..................................................... 16

2.3.2.4 Denial of Service Attacks.............................................................................. 17

2.3.3 Strategies for Breaking into Computers and Networks..........................................18

2.4 SECURITY VULNERABILITY ASSESSMENTS................................................................................ 20

2.4.1 Motivation for Vulnerability Assessment ...................................................................20

2.4.2 Strategies for Vulnerability Assessment .....................................................................21

2.4.3 Vulnerability Assessment Tools ....................................................................................21

2.4.3.1 Nmap.............................................................................................................. 22

2.4.3.2 Nessus.. ........................................................................................................... 23

3. RESEARCH METHODOLOGY ............................................................................................. 27

3.1 RESEARCH GOALS............................................................................................................ 27

3.2 RESEARCH ENVIRONMENT ................................................................................................. 28

3.3 RESEARCH METRICS.......................................................................................................... 28

3.4 RESEARCH PROCEDURES ................................................................................................... 29

3.4.1 Data Gathering .............................................................................................................30

3.4.2 Creating Custom Applications...................................................................................32

3.4.2.1 Nessus Log Parsers (VB6.0 and VBA 6.3). ..................................................... 33

3.4.2.2 Nessus CLI Respawner (Shell Scripting). ....................................................... 34

3.4.2.3 Remote TCP Service Monitor (VB6.0). .......................................................... 35

3.4.2.4 Localhost File Modification Watcher (VB.NET). ........................................... 37

4. PROJECT HISTORY AND FINDINGS.................................................................................... 39

4.1 PROJECT HISTORY ............................................................................................................ 39

4.2 FINDINGS ......................................................................................................................... 42

4.2.1 On the Use of Nessus to Scan Large Networks........................................................42

4.2.2 On the Effect of Service Evolution on Network Security Management.............44

4.2.3 On the Common Sense of Users and Support Personnel......................................45

4.2.4 On the Criticality of Obtaining and Retaining Administrative Support..............46

5. CONCLUSION.................................................................................................................... 49

5.1 GENERAL OBSERVATIONS.................................................................................................. 49

5.2 SPECIFIC RECOMMENDATIONS FOR MAINTAINING NETWORK SECURITY ................................... 51

REFERENCES........................................................................................................................... 54

APPENDIX A: TERMS AND DEFINITIONS................................................................................. 56

APPENDIX B: CREATED SOURCE CODES .............................................................................. 62

NESSUS CLI RESPAWNER (SHELL SCRIPT)..................................................................................... 62

NESSUS LOG PARSER (VB6 AND VBA) ...................................................................................... 64

REMOTE TCP SERVICE MONITOR (VB6) .................................................................................... 72

FILE MODIFICATION WATCHER (VB.NET) ................................................................................... 86

SCRIPT BASED VULNERABILITY SCANNERS.................................................................................... 91

IIS /admin Available ................................................................................................................91

Default Port Anonymous Enabled FTP .................................................................................92

Full Port Sweep Anonymous Enabled..................................................................................93

Nimda Vulnerability .................................................................................................................95

SCRIPT BASED ASSISTANCE APPLICATIONS .................................................................................. 97

Compression and Archival Script .........................................................................................97

Localhost System Log Parser..................................................................................................98

Root Crontab Cronjobs ..........................................................................................................98

Nessus Plugin Information Retriever................................................................................... 100

Red Hat New Login(s) Detector......................................................................................... 101

Red Hat ISO Downloader .................................................................................................... 101

APPENDIX C: PUBLIC SOURCE CODES................................................................................103

AUTOMATICALLY UPDATING NESSUS PLUGINS ............................................................................103

AUTOMATICALLY UPDATING THE NESSUS APPLICATION................................................................109

APPENDIX D: CHARTS, TABLES AND DATA SAMPLES ..........................................................114

OIT ADMINISTRATION WORK LOG SAMPLE................................................................................114

VULNERABILITY ASSESSMENT DATA SAMPLES...............................................................................115

Subnet Computer Vulnerability Output NSR Example .................................................. 115

Subnet Computer Vulnerability Output HTML Sample.................................................. 127

VITA.......................................................................................................................................132

LIST OF FIGURES

PAGE

Figure 1. Output from Nessus NSR Vulnerability Output Parser 34

Figure 2. Remote TCP Service Monitor 36

Figure 3. Remote TCP Service Monitor Options 37

Figure 4. Localhost File Modification Watcher 38

Figure 5. Service Trending Information, 12/2002 thru 2/2004 45

CHAPTER 1

INTRODUCTION

This thesis details a year-long effort to determine and document the

challenges of improving network and computer security at a medium-sized

regional academic institution, East Tennessee State University (ETSU). At the time

of research, the ETSU administrative computer network (151.141.0.0/16) consisted

of 58 subnets containing approximately 3,300 total nodes which are

administered and maintained by the ETSU Office of Information Technology

(OIT) staff. The 3,300 nodes include server and workstation computers, network

printers, routers, hubs, switches, wireless access points and personal digital

assistants. ETSU has approximately 11,000 students and 3,000 faculty and

administrative staff.

Originally, the research goal was to conduct a more extensive case study

of network security vulnerability remediation than the one presented here. The

initial proposal involved a systematic attempt to detect, and address, all readily

detectable vulnerabilities on campus. This work was to include a monitoring of

all costs associated with performing said research and remediation. Performing

these tasks required local or domain level administrative credentials, which I was

originally promised but later denied, due to my student enrollment and lack of

full-time ETSU-OIT employee status. The study that was actually performed

focused on identifying specific high criticality level vulnerabilities present on the

ETSU administrative network and observing the challenges that hindered the

vulnerability assessment and remediation processes.

The data for this research was gathered using a mixture of node-specific

and campus-wide vulnerability sweeps using the open source vulnerability

scanner Nessus (versions 1.2.6 thru 2.1.0) [http://www.nessus.org], and the open

source network exploration application Nmap (versions 3.00 beta thru 3.50)

[http://www.insecure.org/nmap]. Each application was compiled, maintained,

routinely updated, and used on a Red Hat Linux (versions 7.2 thru 9.0)

[http://www.redhat.com] system which I administered. The Nessus and Nmap

scans started in May 2002 and ended in June 2003, with a supplemental Nmap

service discovery sweep in February, 2004.

1.1 Challenges Encountered

This study confirmed four standard observations about the challenges of

implementing network security in large computing environments. These

observations address the issues of scale, automation, process management,

and organizational roadblocks encountered at a management level.

• Domain level vulnerability scanning is a massive task that is best

performed in parallel by multiple servers and supported by

automated data analysis. On average, a complete, sequential

scan using one Nessus server of a single host took approximately 16

minutes to complete. A sweep of the ~3,300 node ETSU network

took 10 days, and generated over 21,000 lines of vulnerability report

data. One of this thesis’s contributions, a set of supplemental

applications for summarizing Nessus vulnerability detail output,

digested and created a report using the full campus sweep data in

approximately 10 minutes versus the estimated 17 hours it would

have taken to review the data manually.

• Securing a network is difficult, due to an ever-increasing

proliferation of networked hosts and default-install services, and an

ever-increasing queue of vulnerabilities to search for and repair.

The number of vulnerable Telnet, FTP, and HTTP servers increased by

approximately 100 every six months during this project, primarily due

to the increasing amount of embedded administration user

interfaces in hardware devices such as routers and switches and

default-install services in standard Windows and Linux operating

system installations. During the research and remediation tasks, the

number of high-risk vulnerabilities detected by the Nessus sweeps

decreased 26.1% from 1,211 to 895. However, during this same

period the total number of vulnerabilities that Nessus scanned for

increased from 933 to 2,088, or 223.8%. While these quantitative

results do not show a complete security hardening of the campus,

they do show an overall decrease of high-risk vulnerabilities directly

related to the work performed. These high-risk vulnerabilities were

affecting 36.7% of campus at the beginning of the research, and

27.1% of campus at the end, even with thirteen months of new

vulnerabilities becoming known and vulnerability plug-ins being

created by the Nessus community and being made public.

• Failures of common sense are still among the most critical problems

for administrators to address. One operational Trojan horse, 15

administration accounts with null or extremely simplistic passwords,

and over 800 unnecessary or unknown services were found during

the initial campus vulnerability sweep.

• Failing to retain organizational support for the duration of the

security remediation process can cause the entire project to come

to a complete standstill. Problems with organizational support,

which occurred several times during this research, included failure

to keep an agreement to grant administrator credentials needed to

facilitate the original study, and inconsistencies in operating

procedures and expectations created by a turnover in

management personnel, due to an insourcing of ETSU’s IT function.

This research’s primary contribution is the evidence it presents to support

the preceding conclusions, evidence that should prove useful to IT

management, system, network, and application administrators in the computer

and network security field.

Malicious computer hackers use academic institutions, which are

perceived as lightly funded organizations which rarely pursue prosecutions, as

training grounds to hone their skills and enhance their prestige among peers. I

believe that the challenges encountered, the applications created and the

data gathered while addressing computer, network and application

vulnerabilities at ETSU will mirror those faced by businesses, home computer

users, and other academic institutions, and will prove valuable for forecasting

required security hardening and remediation costs across various domains.

The remainder of this thesis is divided into four chapters: Chapter 2,

Background, describes the idea origins of this research, background security

research performed, details specific threats and strategies for network security,

and discusses the vulnerability assessment strategies and commercial tools used

for this research. Chapter 3, Research Methodology, describes the goals,

network and social environment, metrics, data gathering procedures and

software tools created for use in this research. Chapter 4, Project History and

Findings describes early events, initial lessons learned from the security

hardening and remediation process, presents the study’s key conclusions,

together with the evidence for these findings. Chapter 5, Conclusion,

summarizes the work performed, and presents a list of final recommendations

and the eventual actions taken by site management.

CHAPTER 2

BACKGROUND

This chapter describes the idea origins for this research, security specific

background research performed, details specific threats and strategies for

computer and network security, and discusses the vulnerability assessment

strategies and commercial tools used for this research.

2.1 Idea Origins

The idea for this research was born from my underlying interest and future

employment hopes in network and computer security and from multiple

conversations with my original thesis advisor, Dr. Phil Pfeiffer IV. One of these

conversations alerted me to a related thesis by fellow graduate student, James

P. Ashe (Jim). Jim was using the open source vulnerability assessment

application Nessus to profile and assess the ETSU administrative network. His

data described the number of vulnerabilities found per host, and detailed the

highest risk protocols, operating systems, applications and services. His data,

when combined with mine, could be used to show a computer security

hardening of the ETSU administrative network over a 3 year period.

2.2 Computer and Network Security Background

Computer, network and application security are growing in importance,

thanks to the increasing value of electronic data and that data’s impact on

and permeation into the public domain [Reynolds]. Computer security is “the

process of preventing and detecting unauthorized use of a computer”

[Howard]. A comparable definition holds for network and application security,

even though each is a distinct entity with different vulnerabilities.

The increasing importance of security affects businesses, the private

sector and academia in strikingly similar ways, which should dovetail with

challenges encountered in said domains and led to the research presented

here. Academic institutions, in particular, have historically been negligent in

security practices, allowing hackers to gain control of portions of their networks

through fairly simple means [Schwartau]. A lack of funding is often cited as a

key reason for inattention to security in academia. The research results

presented here could be used to support grants for further research into

academic network security, for increases of security budgets, and for educating

the general public about the severity of security threats to academic networks.

The focus of this research is virtual security: the task of securing information

stored in computers; messages transmitted through a computer network; and

resources accessed through on-line commands [Schneier]. Virtual security is

distinguished from physical security, which involves limiting physical access to a

system’s hardware. Physical security, although an important part of overall

information security, is not a focus of this research.

2.3 Threats and Fixes to Network and System Security

2.3.1 Vulnerability Definition

A security vulnerability is commonly defined as anything that offers a

potential avenue of attack against a system or network [TECS], for example,

viruses, improper system or application configurations, application backdoors,

and Trojan horses. In this research, the term vulnerability is used in a more limited

sense, to refer to flaws in a program’s configuration or logic that allow an

attacker to receive escalated privileges, disable application or service

operation or compromise computer data; that have been publicized in a wellknown

listing of vulnerability reports; and that can be successfully neutralized,

using a publicly released fix, patch, or workaround. This more specific definition

of vulnerability is consistent with current expectations of system administrators,

who, as a rule, apply available vulnerability fixes (vulnerability remediation)—

and don’t create them. This remediation process typically includes changing

file permissions, applying a group or domain level policy, setting an Operating

System configuration option or installing an available patch.

2.3.2 Specific Threats to Network Security

2.3.2.1 Cleartext Network Transmission. The basic unit of information

transfer in computer networks is the packet: a fragment of control or data

information whose size depends on the underlying network technology.

Numerous applications have been created that passively intercept and copy all

network traffic on a system, server, router or firewall [Chirillo]. These applications,

called protocol analyzers or sniffers, obtain access to network traffic by

exploiting the underlying network’s topology or by masquerading as a network

routing hardware device. Since computer systems commonly exchange

messages in cleartext [Steinke], protocol analyzers allow potential attackers to

intercept and alter information that they are not entitled to access, including

user authentication data.

2.3.2.2 Misconfigured and/or Unpatched Applications. Most

manufacturers of applications and operating systems issue fixes for potential

security holes in response to known exploits [Schwartau]. As a network security

“best practice”, these patches and hot fixes should be installed as they become

available, by the Information Technology support staff [Limoncelli].

Unfortunately, standard home users, small business operators and universities

generally lack the technical proficiency or budget to apply patches as they are

released. This inability to keep current is the primary reason malicious hackers

have opportunities to penetrate a computer or network host.

2.3.2.3 Well-known or Default Install Accounts. Well-known accounts,

those that applications create or support by default, are standard avenues for

system or network compromise. These accounts include ‘root’, ‘admin’, ‘guest’,

‘administrator’, ‘sa’, and ‘operator’ to name a few. Another best practice in

network and computer security is to rename these well-known accounts for

administrative use, and then to create unprivileged, “dummy” accounts in their

place [Technet]. Activity involving the newly created dummy ‘root’, ‘admin’,

‘guest’, ‘administrator’, ‘sa’, or ‘operator’ accounts can then be monitored

using application, system, and network logs as a security check for attempted,

non-authorized intrusions. The combination of these “best practices” of network

security, when implemented successfully, constitute host or network defense-indepth.

2.3.2.4 Denial of Service Attacks. Denials of Service (DoS), Distributed

Denial of Service (DDoS), and Out of Band (OoB) attacks have become

common because they are easy and inexpensive to implement [Schneier].

Such attacks hurt their victims by monopolizing the target’s resources. DoS and

DDoS attacks consume resources on the target machine or network, impeding

their normal operation. Out of Band attacks use oversized, malformed,

fragmented or otherwise specifically altered individual or packet streams to

disrupt the target machine’s TCP stack.

DoS and DDoS attacks, although primitive in complexity, can be

devastating in the real world [Scambray]. An example DoS/DDoS attack is the

CHARGEN attack on Microsoft’s SQL Ping service. This service, which is installed

by default with Microsoft SQL Server 2000, allows a client to ping a port of the

client’s choice. Hackers can disable a host that supports this service by creating

simple scripts that continually request this service to ping a CHARGEN service on

another machine. Every custom-crafted packet sent to CHARGEN causes

millions of packets to be directed to the victim’s machine. A few compromised

machines, or zombies, running these scripts are enough overwhelm the SQL

server’s ability to respond, thereby disabling the database service. Fortunately,

Microsoft implemented the SQL ping service running on a UDP port, which differs

from a TCP port and does not require a full end to end connection. Because of

this UDP implementation, more attempts are needed to disrupt the service. Still,

the service is installed by default on SQL Server 2000 and creates a DoS target.

The innate problem with defending against DoS, DDoS, OoB and flood

attacks is their inherent simplicity. Script Kiddies—novice hackers who depend

on other people’s scripts to perform attacks—can effectively disrupt or disable

access to network or computer resources without the level of technical

practical understanding usually associated with hacking attacks. These styles of

attacks can vary from the small and mildly annoying case of tying up a shared

printer or resource, to the extremely costly disruption of an enterprise level web

or database server.

2.3.3 Strategies for Breaking into Computers and Networks

Strategies for breaking into single computers and networks of computers

are similar. In most cases, hackers who target an unfamiliar network or

computer will footprint (i.e., scout) it to determine the network structure,

defenses and possibly the hardware and software in use [Scambray].

Footprinting can involve rummaging through trash for discarded data, using

network transmissions to determine the operating system or services in use, and

searching news groups for informative email from the target organization

[Scambray]. A relatively new form of footprinting involves web log (blog)

searches for employee online journals. Blogs, like newsgroup postings and the

other footprinting sources described, can betray the target organization’s

underlying policies, procedures, applications and hardware.

Footprinting can occur from inside and outside a network, although

footprinting from outside is the most common [Schwartau]. A key objective of a

footprinting session is to discover a system’s or network’s open ports and the

privilege levels of the applications that “own” these ports. Open ports are

analogous to open doors into the target system or network. Every open port is

associated with a program that “owns” it: i.e., responds to requests directed

toward that port. Microsoft SQL Server 2000 runs by default on port 1433. Most

web servers run by default on port 80. Almost all services have an associated

default port. The level of privilege of the program that owns a given port

determines the damage that hacker that subverts that port can do. If, for

example, a port is open and the attached service is running as user “root” or

“administrator”, a compromise of the service attached to that port will afford a

hacker high level root or administrative access to the underlying system or

network device.

Footprinting would be used by an intruder to discover services available

to the public. An administrator discussing current projects on a newsgroup, a

faculty member posting their thoughts on the current mail server roll-out, or

simple port scanning of the academic network IP range can advise the hacker

of the application/server version and patch levels, and from that point the trivial

next step to research available vulnerabilities for the services found on the

academic network. Entry level hackers use Nessus to profile a target, and get

an extremely descriptive readout on potential areas of attack. This would

generate a large signature easily tracked by administrators by reviewing access

log files, but generally isn’t watched for by Universities due to the previously

discussed budgetary concerns found in academia.

2.4 Security Vulnerability Assessments

2.4.1 Motivation for Vulnerability Assessment

Information security authorities advise administrators to conduct regular

assessments of their networks’ vulnerability level, as a starting point for

discovering and repairing known vulnerabilities [Bott]. Since networked

communication is managed through a system’s network ports, those ports are

the starting point for checking for vulnerabilities. An administrative awareness of

a system or network’s open ports, vulnerabilities, and services also increases the

ability to determine available avenues for attack or even when an attack may

be underway.

2.4.2 Strategies for Vulnerability Assessment

The two principal strategies for assessing security vulnerabilities are black

box [Webopedia] and crystal box [Lam] assessment. In a black box assessment,

the security auditor performs a stock set of network-based probes and scans on

the target domain; no inside knowledge of the target computer or network

environment is assumed. This style of probing and scanning simulates the

footprinting performed by an external hacker with a limited knowledge of the

target system. In a crystal box vulnerability assessment, the intrusion test team is

provided with insider knowledge of the system to be tested, typically in the form

of network diagrams and system names, IP addresses, platforms, and lists of

services and datasets hosted by the network’s devices. This testing strategy

simulates an attack by a disgruntled employee, or a collusion situation involving

multiple attackers with insider information.

2.4.3 Vulnerability Assessment Tools

At the time of writing, there is no single vulnerability assessment

application that performs all of the different audits recommended by authorities

on network security. Accordingly, system and network administration best

practice involves the use of multiple scanning applications and techniques to

help ensure network security. The work described in this thesis relied on two

commonly used applications for assessing network security: the Nmap port

scanner and the Nessus vulnerability auditor.

2.4.3.1 Nmap. Nmap is a port scanning security auditor used in both

black and crystal box assessment that supports the use of three strategies for

footprinting hosts. Nmap can ping a set of hosts to determine which hosts are

alive, and scan a host’s ports to determine what services it supports. Nmap can

also use a database of characteristics to determine what operating system a

host is running. By default, Nmap performs a ping sweep to find live hosts and

scans their ports, using the standard services-to-port reference matrix included

with the Nmap installation. Nmap can scan network hosts using one of six

methods: TCP connect() scans, TCP SYN scans, stealth FIN scans, Xmas tree

scans, Null scans, UDP scans, and ping scans.

Nmap identifies services using its service-to-port association matrix. This

allows some margin of error, as a FTP server (default port 21) configured to run

on port 23 (the default port for Telnet) will return as a Telnet server in Nmap. This

use of non-standard ports is typically motivated by a desire to evade firewall

controls—ISP’s, for example, routinely block port 80, prompting some users of

broadband Internet services to run web servers on port 8080—or to disguise a

service’s presence. Using non-standard ports to enhance system security is not

considered a best practice: a determined attacker will locate these services,

using techniques like complete port scanning and banner enumeration and

banner recognition [Scambray].

2.4.3.2 Nessus. The Nessus vulnerability assessment tool was first released

to the public in April of 1998 [Kooij]. At the time of research, Nessus is not

funded by any corporate entities or committees, and is maintained as a

community service by its authors and a small group of Nessus enthusiasts. Nessus

...

The Challenges of Network Security

This is a preview of the whole essay

Document Details

Related Essays

Wireless network security

NETWORK BASED DIFFUSION OF KNOWLEDGE IN MEDICINE

Types of IT Security Threats

The role of IT Security in a corporation