Unit-V: Data Storage

 Unit-V: Data Storage 

Magnetic disk, magnetic tape, CD-ROM, WORM, Optical disk, mirrored disk, fault 

tolerance, RAID, RAID-Disk network interface cards. Network protection devices, Power 

Protection Devices, UPS, Surge protectors. The future of client server Computing Enabling 

Technologies, The transformational system. 

Magnetic disk

What Does Magnetic Disk Mean? 

A magnetic disk is a storage device that uses a magnetization process to write, rewrite and access data. It is covered with a magnetic coating and stores data in the form of tracks, spots and sectors. Hard disks, zip disks and floppy disks are common examples of magnetic disks. 

Advertisement 

Techopedia Explains Magnetic Disk 

A magnetic disk primarily consists of a rotating magnetic surface (called platter) and a mechanical arm that moves over it. Together, they form a “comb”. The mechanical arm is used to read from and write to the disk. The data on a magnetic disk is read and written using a magnetization process. 

The platter keeps spinning at high speed while the head of the arm moves across its surface. Since the whole device is hermetically sealed, the head floats on a thin film of air. When a small current is applied to the head, tiny spots on the disk surface are magnetized and data is stored. Vice-versa, a small current could be applied to those tiny spots on the platter when the head needs to read the data. 

Data is organized on the disk in the form of tracks and sectors, where tracks are the circular divisions of the disk. Tracks are further divided into sectors that contain blocks of data. All read and write operations on the magnetic disk are performed on the sectors. The floating heads require very precise control to read/write data due to the proximity of the tracks. 

Early devices lacked the precision of modern ones and allowed for just a certain number of tracks to be placed in each disk. Greater precision of the heads allowed for a much greater number of tracks to be closely packed together in subsequent devices. Together with the invention of RAID (redundant array of

inexpensive disks), a technology that combines multiple disk drives, the storage capacity of later devices increased year after year. 

Magnetic disks have traditionally been used as secondary storage devices in computers, and represented the mainstream technology for decades. With the advent of solid-state drives (SSDs), magnetic disks are no longer considered the only option, but are still commonly used. 

The first magnetic hard drive built by IBM in 1956 was a large machine consisting of 50 21-inch (53-cm) disks. Despite its size, it could store just 5 megabytes of data. Since then, magnetic disks have increased their storage capacities many times-folds, while their size has decreased comparably. 

The size of modern hard disks is just about 3.5 inches (approx. 9 cm) with their capacity easily reaching one or more terabytes. A similar fate happened to floppy disks, which shrunk from the original 8 inches of the late 60s, to the much smaller 3.5 inches of the early 90s. However, floppy disks have eventually became obsolete after the introduction of CD-ROMs in the late 1990s and now have all but completely disappeared.

magnetic tape

magnetic recording, method of preserving sounds, pictures, and data in the form of electrical signals through the selective magnetization of portions of a magnetic material. The principle of magnetic recording was first demonstrated by the Danish engineer Valdemar Poulsen in 1900, when he introduced a machine called the telegraphone that recorded speech magnetically on steel wire. 

In the years following Poulsen’s invention, devices using a wide variety of magnetic recording mediums have been developed by researchers in Germany, Great Britain, and the United States. Principal among them are magnetic tape and disk recorders, which are used not only to reproduce audio and video signals but also to store computer data and measurements from instruments employed in scientific and medical research. Other significant magnetic recording devices include magnetic drum, core, and bubble units designed specifically to provide auxiliary data storage for computer systems. 

Magnetic tape devices. Magnetic tape provides a compact, economical means of preserving and reproducing varied forms of information. Recordings on tape can be played back immediately and are easily erased, permitting the tape to be reused many times without a loss in quality of recording. For these reasons, tape is the most widely used of the various magnetic recording mediums. It consists of a narrow plastic ribbon coated with fine particles of iron oxide or other readily magnetizable material. In recording on tape, an electrical signal passes through a recording head as the tape is drawn past, leaving a magnetic imprint on the tape’s surface. When the recorded tape is drawn past the playback or reproducing head, a signal is induced that is the equivalent of the recorded signal. This signal is amplified to the intensity appropriate to the output equipment.

CD-ROM,

CD-ROM, abbreviation of compact disc read-only memory, type of computer memory in the form of a compact disc that is read by optical means. A CD-ROM drive uses a low-power laser beam to read digitized (binary) data that has been encoded in the form of tiny pits on an optical disk. The drive then feeds the data to a computer for processing. The standard compact disc was introduced in 1982 for digital audio reproduction. But, because any type of information can be represented digitally, the standard CD was adapted by the computer industry, beginning in the mid-1980s, as a low-cost 

storage-and-distribution medium for large computer programs, graphics, and databases. With a storage capacity of 680 megabytes, the CD-ROM found rapid commercial acceptance as an alternative to so-called floppy disks (with a maximum capacity of 1.4 megabytes). Britannica Quiz Computers and Technology Quiz 

Unlike conventional magnetic storage technologies (e.g., tapes, floppy disks, and hard disks), CDs and CD-ROMs are not recordable—hence the tag “read only.” This limitation spurred the development of various recordable magnetic-optical hybrid storage devices; but they generally failed to penetrate beyond the publishing world, where large multimedia files are regularly exchanged, because of incompatibility with standard CD and CD-ROM players. In the early 1990s a new type of CD became available: CD-Recordable, or CD-R. These discs differ from regular CDs in having a light-sensitive organic dye layer which can be “burned” to produce a chemical “dark” spot, analogous to an ordinary CD’s pits, that can be read by existing CD and CD-ROM players. Such CDs are also known as WORM discs, for “Write Once Read Many.” A rewritable version based on excitable crystals and known as CD-RW was introduced in the mid-1990s. Because both CD-R and CD-RW recorders originally required a computer to operate, they had limited acceptance outside of use as computer software and data backup devices. 

To handle the proliferation of ever-larger multimedia files (audio, graphic, and video) in computer games, educational software, and electronic encyclopaedias—as well as high-definition movies for television entertainment systems—an expanded storage medium, the digital videodisc (DVD), was introduced in 1995, and a storage medium with even more capacity, Blu-ray, was introduced in 2002. However, with the increased storage capacity of computers and the easy distribution of large files over the Internet, the use of CD-ROMs declined in the 21st century.

WORM

What Is a Worm Virus? 

Contact Us 

A worm virus refers to a malicious program that replicates itself, automatically spreading through a network. In this definition of computer worms, the worm virus exploits vulnerabilities in your security software to steal sensitive information, install backdoors that can be used to access the system, corrupt files, and do other kinds of harm. 

Worms consume large volumes of memory, as well as bandwidth. This results in servers, individual systems, and networks getting overloaded and malfunctioning. A worm is different from a virus, however, because a worm can operate on its own while a virus needs a host computer. 

Classifications and Names of Worms 

Email-Worm 

An email-worm refers to a worm that is able to copy itself and spread through files attached to email messages. 

IM-Worm 

An Instant Messenger (IM) worm is a kind of worm that can spread through IM networks. When an IM-worm is operating, it typically finds the address book belonging to the user and tries to transmit a copy of itself to all of the person’s contacts. 

IRC-Worm

An IRC-worm makes use of Internet Relay Chat (IRC) networks to send itself over to other host machines. An IRC-worm drops a script into the IRC’s client directory within the machine it infects. 

Net-Worm 

A net-worm refers to a kind of worm that can find new hosts by using shares made over a network. This is done using a server or hard drive that multiple computers access via a local-area network (LAN). 

P2P-Worm 

A P2P-worm is spread through peer-to-peer (P2P) networks. It uses the P2P connections to send copies of itself to users. 

How Do Worm Virus/Computer Worms Work and Spread? 

To get a worm in a computer, the worm is often transmitted through vulnerabilities in software. They could also be sent through email attachments or within instant messages or spam emails. After a file is opened, it may link the user to a malicious website or it could download the worm to the user’s device automatically. After the worm is on the device, it infects it without the user being able to tell. 

Worms have the ability to delete and modify files. They can also inject more malicious software into a workstation or other device. Sometimes, the worm’s primary mission is to replicate itself again and again—simply to waste system resources, like bandwidth or hard drive space. Worms can also steal sensitive data and pave a way for a hacker to get into the computer by installing a backdoor they can access. 

Files Sent as Email Attachments 

The user clicks on a file attached to an email and the worm is activated. Via a Link to a Web or FTP Resource 

When the user clicks a link to a web or File Transfer Protocol (FTP) resource, the worm is automatically downloaded to their machine. 

Via a Link Sent in an ICQ or IRC Message 

An I Seek You (ICQ) or IRC message can contain a link to a worm, which, when clicked, can install the worm on the user’s device. 

Through Network Packets

Network packets can penetrate into the computer’s memory. At that point, the worm gets activated, infecting the host computer. 

Via Peer-to-Peer (P2P) File-sharing Networks \

Optical disk

What Does Optical Disk Mean? 

An optical disk is any computer disk that uses optical storage techniques and technology to read and write data. It is a computer storage disk that stores data digitally and uses laser beams (transmitted from a laser head mounted on an optical disk drive) to read and write data. 

Advertisement 

Techopedia Explains Optical Disk 

An optical disk is primarily used as a portable and secondary storage device. It can store more data than the previous generation of magnetic storage media, and has a relatively longer lifespan. Compact disks (CD), digital versatile/video disks (DVD) and Blu-ray disks are currently the most commonly used forms of optical disks. These disks are generally used to: 

● Distribute software to customers 

● Store large amounts of data such as music, images and videos ● Transfer data to different computers or devices 

● Back up data from a local machine 

mirrored disk,

What Does Disk Mirroring Mean?

Disk mirroring is a technique used to protect a computer system from loss of data and other potential losses due to disk failures. In this technique, the data is duplicated by being written to two or more identical hard drives, all of which are connected to one disk controller card. If one hard drive fails, the data can be retrieved from the other mirrored hard drives. 

Disk mirroring if often referred to as RAID 1 or RAID Level 1. 

Techopedia Explains Disk Mirroring 

Disk mirroring is a form of disk backup in which anything that is written to a disk is simultaneously written to a second disk. This creates fault tolerance in the critical storage systems. If a physical hardware failure occurs in a disk system, the data is not lost, as the other hard disk contains an exact copy of that data. 

Mirroring can be either hardware or software based. 

Hardware-based mirroring is implemented through the use of RAID controllers installed in the system to which separate hard disk drives are attached. These hard disks appear as different volumes to the system. Each data sector is identically written to all the volumes, thus creating multiple copies of the volumes. At the expense of mild system performance degradation, fault tolerance is introduced to the system. 

Software-based mirroring requires certain mirroring applications to be installed in the system. The software-based mirroring solution is usually less expensive and more flexible, but it results in more system performance degradation and is more susceptible to incompatibilities like boot time problems. 

A popular alternative to disk mirroring is disk striping, in which data is striped in blocks over multiple volumes (disks). In case of a failure, the failed disk is recreated with the help of checksum or other data present on other disks. Unlike disk mirroring, disk striping may not fully recover lost data.

Fault tolerance

What is fault tolerance 

Fault tolerance refers to the ability of a system (computer, network, cloud cluster, etc.) to continue operating without interruption when one or more of its components fail. 

The objective of creating a fault-tolerant system is to prevent disruptions arising from a single point of failure, ensuring the high availability and business continuity of mission-critical applications or systems. 

Fault-tolerant systems use backup components that automatically take the place of failed components, ensuring no loss of service. These include: 

● Hardware systems that are backed up by identical or equivalent systems. For example, a server can be made fault tolerant by using an identical server running in parallel, with all operations mirrored to the backup server. 

● Software systems that are backed up by other software instances. For example, a database with customer information can be continuously replicated to another machine. If the primary database goes down, operations can be automatically redirected to the second database. 

● Power sources that are made fault tolerant using alternative sources. For example, many organizations have power generators that can take over in case main line electricity fails.

In similar fashion, any system or component which is a single point of failure can be made fault tolerant using redundancy. 

Fault tolerance can play a role in a disaster recovery strategy. For example, fault-tolerant systems with backup components in the cloud can restore mission-critical systems quickly, even if a natural or human-induced disaster destroys on-premise IT infrastructure. 

RAID

Understanding RAID 

Damage to the enterprise, caused by disk failure, has become a correspondingly heavier burden as disk capacity use has grown. Storage system downtime can mean companies fail to take full advantage of business opportunities because of the management overheads of securing important data.

RAID technology not only prevents such data loss and failure but also enhances business performance. 

This content covers the topics of RAID technology from its processes to use. 

What is RAID? 

RAID is an acronym for "Redundant Array of Inexpensive Disks". When interpreted literally, it means storing information across an array of relatively low cost hard disk drives (HDDs). It is generally considered to be "Technology that combines numbers of such inexpensive HDDs into a single HDD." 

RAID is the use of multiple disks to manage HDD data using a range of different techniques. These are typically divided into 6 levels ; RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5. They all differ in terms of data deployment and the type of redundancy offered. 

Now Fujitsu provides RAID 6 with it's ETERNUS DX and ETERNUS AF. This ensures enhanced reliability through use of double parity. 

The technology was initially proposed by a study group of Professor. Patterson at the University of California. When they presented their paper, the "I" in RAID represented "Inexpensive". This was because they were seeking ways to use the inexpensive HDDs that were on the market, in their study, instead of the proprietary and rather exclusive disk devices commonly used at that time to improve disk drive speed and reliability. 

But today, as expensive dedicated HDDs have all but disappeared from the market, the "I" in RAID has come to represent "Independent". 

It has also become popular to mix and match the various RAID level technologies to provide more specific cost reductions and performance enhancements.

Both software RAID and hardware RAID are available for installation. 

Software RAID can be implemented through features that combine multiple disk devices connected directly to a host computer (typically via a SCSI interface) and regard them as a single logical memory device. This feature introduced with the operating systems Windows NT/2000 is commonly used. 

With Hardware RAID, a control component, independent of the host CPU implements RAID. The two most popular methods of Hardware RAID are via PCI bus connection to the host computer using a card, or integrated with the disk drive and connected to the host computer via fibre channel or SCSI. 

Hardware RAID is by far the most common method in full-fledged server systems, as it places no additional processing burdens on the server. 

Let's look at the details of the different RAID technology levels. RAID leve 

What is RAID Storage? 

RAID – or Redundant Array of Independent Disks – is a type of storage that writes data across multiple drives within the same system. Different configurations are expressed as numbers, such as RAID 0, RAID 1, or RAID 5. Each RAID type gives 

users different benefits — increased performance, greater fault tolerance, or a combination of both — depending on how it writes and distributes your data. 

Pro performance 

Get the performance and protection you need for data-intensive creative workflows.

Production power 

RAID solutions can be configured for the most rigorous uses, such as time-sensitive, data-sensitive post-production workflows. 

Cloud-friendly 

Cloud-ready RAID storage can be ideal for both business and home users that want to access their data from anywhere. 

Fault tolerant 

RAID can deliver high data protection and redundancy whenever uptime and availability are critical needs. 

RAID-Disk network interface cards.

A network interface card (NIC) is a hardware component without which a computer cannot be connected over a network. It is a circuit board installed in a computer that provides a dedicated network connection to the computer. It is also called network interface controller, network adapter or LAN adapter.

Purpose 

● NIC allows both wired and wireless communications. 

● NIC allows communications between computers connected via local area network (LAN) as well as communications over large-scale network through Internet Protocol (IP). 

● NIC is both a physical layer and a data link layer device, i.e. it provides the necessary hardware circuitry so that the physical layer processes and some data link layer processes can run on it. 

Types of NIC Cards 

NIC cards are of two types − 

Internal Network Cards 

In internal networks cards, motherboard has a slot for the network card where it can be inserted. It requires network cables to provide network access. Internal network cards are of two types. The first type uses Peripheral Component Interconnect (PCI) connection, while the second type uses Industry Standard Architecture (ISA).

External Network Cards 

In desktops and laptops that do not have an internal NIC, external NICs are used. External network cards are of two types: Wireless and USB based. Wireless network card needs to be inserted into the motherboard, however no network cable is required to connect to the network. They are useful while traveling or accessing a wireless signal. 

What Are Network Security Devices?

Network Security devices are typically physical or virtualised hardware appliances, with vendor specific software installed. 

Occasionally, businesses purchase commodity server hardware and install custom software to create their own network security device. Depending on your company’s particular needs, one approach may be more cost-effective than the other for a particular type of device. 

With the rise of cloud computing, some devices that would be traditionally hosted on a local network are instead provided by a third party. Businesses commonly host security applications used to protect web applications and email communications in the cloud, especially if the websites and email services themselves are cloud-hosted. 

Types of Network Security Devices 

Network security appliances have lots of different jobs. Some manage network traffic, others detect threats, and still others provide secure remote access. Many security devices combine functionality from multiple other devices, especially those that are intended for smaller businesses. 

Firewalls 

Firewalls are one of the most fundamental network security appliances. Like many other security devices, firewalls can come in hardware or software forms. Most of the time, businesses choose to use dedicated, specialized hardware since it can handle more traffic and has better vendor support. 

Firewalls provide separation between your internal network and the wider Internet. They can block connections on specific ports, from specific IP addresses, and from machines or networks matching other criteria. Most firewalls are configured to deny incoming traffic by default, providing a baseline of security for your network. 

Intrusion Protection Systems (IPS) 

Network-based intrusion protection systems proactively monitor all of the traffic going through your network. Using pre-made profiles, signature detection, artificial

intelligence, and anomaly detection, IPS systems can detect many kinds of network intrusions, from malware on endpoint devices to denial of service attacks. 

One of the most useful features of network-based intrusion protection is that it can talk to firewalls and other network hardware in real time as threats are discovered. As an example, an IPS system could detect a device with malware installed from the unusual and suspicious network traffic it produces. Afterwards, the IPS can request that the firewall quarantines this infected device on its own partitioned subnet so that it is unable to cause further damage. 

Unified Threat Management (UTM) 

In a modern business network, administrators might control a half dozen or more separate network appliances with security functions. If multiple products come from different vendors, managing a quickly-unfolding network threat can be challenging. 

UTMs combine a network firewall, an intrusion detection system, an intrusion prevention system, and other features. For smaller businesses or those without significant IT resources, using a UTM can save lots of time and money. However, UTMs are not always better than discrete equipment: they create a single point of failure that can take down the whole network if something goes wrong. 

Network Access Control 

Keeping infected or insecurely configured endpoint devices off of the corporate network is critical to security. As a result, network access control devices link network authentication with the state of endpoint devices. 

For example, an integrated network access control solution could make sure that devices could not authenticate themselves without having the latest security updates installed. 

Email Security Gateways 

While more and more businesses move to cloud-hosted email solutions, network email gateways can still be useful. These devices monitor incoming and outgoing

email traffic for spam, viruses, phishing attempts, and compromised accounts. Recent, advanced email security gateways also use historical data and statistical analysis to detect anomalies with more accuracy. 

Some vendors sell hardware email security gateways, while others provide services that run on mail servers or alongside cloud-based email hosting. 

Web Application Firewalls (WAF) 

Like a regular network firewall, a WAF selectively allows or blocks traffic based on predefined criteria or suspicious activity. Web applications commonly have security vulnerabilities that can be used to compromise a company’s network and leak data. While finding and fixing all of these issues would be the ideal solution, using a web application firewall is a good next layer of defence. 

A WAF can block URLs and requests containing suspicious payloads, evidence of SQL injection attempts, and other attacks. They can come in the form of a physical device, a software extension to another network security device, or software installed on a standard reverse proxy server. 

VPN Gateways 

With the rise of remote work, every company needs to ensure that their internal network resources are accessible securely from anywhere. A virtual private network or VPN device can help here. In effect, when employees connect to the VPN, their traffic enters the internal network from the VPN device instead of going straight to the Internet. 

In addition to security benefits, VPN gateways give employees access to printers, Intranet sites, and other internal devices, saving time and improving productivity. 

Network Device Backup and Recovery 

With so many individual network devices, applying and rolling back configuration changes can be challenging. Additionally, large numbers of separate devices are difficult to recover quickly in the event of a disaster.

For these reasons, centralised backup and recovery for network devices is very useful. Network configuration management tools automate the backup process by securely storing the configuration and state of network devices, simplifying rollback or restore operations. 

Electrical Protection and Control 

All electrical circuits and equipment need to be protected from damages due to abnormal currents and voltage fluctuations. Protection devices ensure that the current does not exceed the withstanding current limit of your equipment. This is imperative for not only the electrical equipment but also to protect your family, house and office from any mishap. For homes and small buildings, Schneider Electric India offers a range of devices that can help make the electrical installations in your home and offices safer, more reliable and always under control. 

Our electrical protection systems are carefully designed to protect your living and workspaces from electrical failures and mishappenings. All Schneider products offer the best-in class customer experience and are created to cater to all electrical protection needs, from protecting people and your electrical equipment to preventing accidents like short circuits and fires. It can all be made possible with our diverse range of miniature circuit breakers, surge protection devices, residual current devices, accompanied by our contemporary, elegant, and easy to install electrical enclosures that compliment modern style built spaces. With our products, you can sit back and not worry about your appliances and circuits. 

Electrical protection devices are applied to existing electrical equipment to prevent any mishaps or abnormal functioning. These devices can identify and address unacceptable problems and take necessary corrective action. Some examples of electrical protection devices are lightning arresters, surge protectors, fuses, relays, circuit breakers, reclosers, and other devices.

Every electrical circuit has a maximum voltage or amperage. If this value is exceeded, the wire will overheat, causing the wire insulation to melt and fire to ignite. Electrical protection devices are important because they help protect the occupants and the infrastructure that could incur significant damages caused by electrical accidents. Overloading, short-circuiting, and earth faults are all examples of unintentional electrical accidents that occur without warning. As a result, electrical protection devices must be used in every family, company, institution, and factory. 

(Uninterruptible Power Supply) A device that provides battery backup when the electricafails or drops to an unacceptable voltage level. Small UPS systems provide power for a minutes; enough to power down the computer in an orderly manner, while larger system

enough battery for several hours. In mission critical datacenters, UPS systems are usedfew minutes until electrical generators take over. 

UPS systems can be set up to alert file servers to shut down in an orderly manner when outage has occurred, and the batteries are running out. 

Surge Suppression and Voltage Regulation 

A surge protector filters out surges and spikes, and a voltage regulator maintains uniforduring a brownout, but a UPS keeps a computer running when there is no electrical powsystems typically provide surge suppression and may provide voltage regulation. See susuppression. 

Standby and Line Interactive 

A standby UPS, also called an "offline UPS," is the most common type of UPS found in a or office supply store. It draws current from the AC outlet and switches to battery withinmilliseconds after detecting a power failure.

Surge protectors.

When you put together a computer system, one piece of standard equipment you'll probably buy is a surge protector. They serve one obvious function; they let you plug multiple components into one power outlet. With all the different components that make up a computer system, surge protectors are useful devices. 

But the other function of a surge protector power strip — protecting the electronics in your computer from power surges — is far more important. So how do surge protectors work and when you need them? And how do you know if the one you have is good quality? 

The main job of a surge protector system is to protect electronic devices from "surges." So if you're wondering what a surge protector does, the first question is, "What are surges?" And then, "Why do electronics need to be protected from them?" 

A power surge, or transient voltage, is an increase in voltage significantly above the designated level in a flow of electricity. In normal household and office wiring in the United States, the standard voltage is 120 volts. If the voltage rises above 120 volts, there is a problem, and a surge protector helps prevent surges from damaging your electronics. 

To understand the problem, it is helpful to understand something about voltage. Voltage is a measure of a difference in electric potential energy. Electric current travels from point to point because there is a greater electric potential energy on one end of the wire than there is on the other. This is the same sort of principle that makes water under pressure flow out of a hose — higher pressure on one end of the hose pushes water toward an area of lower pressure. You can think of voltage as a measure of electrical pressure. 

Various factors can cause a brief increase in voltage:

● When the increase lasts three nanoseconds (billionths of a second) or more, it's called a surge. 

● When it only lasts for one or two nanoseconds, it's called a spike. 

If the surge or spike is high enough, it can cause major damage on a machine. The effect is very similar to applying too much water pressure to a hose. If there is too much water pressure, a hose will burst. A similar thing happens when too much electrical pressure runs through a wire — the wire "bursts." Actually, the wire heats up like the filament in a light bulb and it burns, but it's the same idea. 

Even if increased voltage doesn't immediately break your electronics, it may put extra strain on the components, wearing them down over time. In the next section, we'll look at what surge protectors do to prevent this from happening. 

The future of client server Computing Enabling 

Technologies, 

The Future of Client/Server Computing 

Executive Summary

The single-system image is a reality. In the future, cheap and powerful workstation 

technology will be available to everyone—with truly distributed applications using processing power wherever it is available and providing information wherever it is needed. In the future, information will be available for use by owners and authorized users, without the constant need for professional systems developers and their complex programming languages. The future will bring information captured at its source and available immediately to authorized users. 

The future will provide information from data in its original form: image, video, audio, graphics, document, spreadsheet, or structured data, without the need to be aware of specific software for each form. Successful organizations of the future—those that are market-driven and competitive—will be ones using client/server as an enabling technology to add recognized value to their product or service. The future is now for early adopters of technology. By the turn of the century, the enterprise on the desk will be the norm for all successful organizations. Laggards will not be price competitive, will not provide competitive customer services, and soon will cease to exist. 

Enabling Technologies 

Client/server computing describes a model for building application systems, along with the core hardware and software technology that helps in building these systems. The material in the following paragraphs describes aggregations of these core technologies that have created enabling technologies. Enabling technologies are combinations of hardware and software that can be used to assist in creating a particular kind of application system.

The transformational system. 

The Transformation System (TS) is used to automatise common tasks related to production activities. Just to make some basic examples, the TS can handle the generation of Simulation jobs, or Data Re-processing jobs as soon as a ‘pre-defined’ data-set is available, or Data Replication to ‘pre-defined’ SE destinations as soon as the first replica is registered in the Catalog. 

The lingo used here needs a little explanation: throughout this document the terms “transformation” and “production” are often used to mean the same thing: 

● A “production” is a transformation managed by the TS that is a “Data Processing” transformation (e.g. Simulation, Merge, 

DataReconstruction…). A Production ends up creating jobs in the WMS. ● A “Data Manipulation” transformation replicates, or remove, data from storage elements. A “Data Manipulation” transformation ends up creating requests in the RMS (Request Management System).

For each high-level production task, the production manager creates a 

transformation. Each transformation can have different parameters. The main parameters of a Transformation are the following: 

● Type (e.g. Simulation, DataProcessing, Removal, Replication) ● Plugin (Standard, BySize, etc.) 

● The possibility of having Input Files. 

Within the TS a user can (for example): 

● Generate several identical tasks, differing by few parameters (e.g. Input Files list) 

● Extend the number of tasks 

● have one single high-level object (the Transformation) associated to a given production for global monitoring 

Disadvantages:

● For very large installations, the submission may be perceived as slow, since there is no use (not yet) of Parametric jobs. 

New in version v6r20p3: Bulk submission of jobs is working for the transformations, so job submission can be sped up considerably. 

Several improvements have been made in the TS to handle scalability, and extensibility issues. While the system structure remains intact, “tricks” like threading and caching have been extensively applied. 

It’s not possible to use ISB (Input Sandbox) to ship local files as for ‘normal’ Jobs (this should not be considered, anyway, a disadvantage). 

8.1. Architecture 

The TS is a standard DIRAC system, and therefore it is composed by components in the following categories: Services, DBs, Agents. A technical drawing explaining the interactions between the various components follow.

● Services 

○

 What Is Transformation Server?

This chapter is an introduction to Transformation Server. Transformation Server is an add-on component to the Outside In Export SDKs. It provides an alternative means of controlling Outside In, by supplying a runtime environment that manages file-export operations in processes that execute independently of your application. In other words, it moves the execution of Outside In from an in-process component model to a client-server model.

Major features of Transformation Server include:

• Service-oriented architecture that allows your application to control export operations with complete isolation from the memory and execution space of the export process, for maximum fault tolerance.

• Multiple interfaces - Transformation Server includes interfaces in C/C++, Java, or the SOAP protocol.

• Process management - Transformation Server can monitor its export processes and will restart them in the event of an exception or an infinite loop.

• Support for all of the Outside In Export SDKs.

• A published add-on interface that allows the developer to implement custom input/output.

• A published add-on interface that allows customer or third- party export code to be integrated into Transformation Server.