IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

An IBM Redbook Publication
IBM Redbook Form Number: SG24-5757-04
ISBN: 0738490326
ISBN: 9780738490328
Publication Date: 22-Jul-2004
Last Update Date: 25-Jan-2006
Find Similar Download

Related People

Cathy Warrick - Author [+3] [-3]
Paul Jenkin - Author
Claus Schiefer - Author
Jiri Spacek - Author

Abstract

This IBM Redbooks publication describes the copy functions available with the IBM TotalStorage Enterprise Storage Server (ESS). The powerful ESS Copy Services functions are explained in detail, and their characteristics are thoroughly covered. This book also provides information on how to manage the various ESS Copy Services functions, and discusses their implementation.

Because this book provides a broad understanding of the ESS Copy Services functions, as well as presenting details about the management interfaces and the implementation considerations, it is a recommended manual for IT professionals who are planning the implementation of any of the ESS Copy Services functions in an open systems environment, as well as for those who will manage these environments.

This fifth edition of the book has been updated with the latest ESS Copy Services announcements: Asynchronous PPRC for a two site Disaster Recovery solution, a FlashCopy target can now be a PPRC primary, and other updates associated with ESS LIC 2.4.0. These include the ESS Model 750 overview, minor additions for ESS API support, Multiple Device Manager Replication Manager, and the Asynchronous PPRC Utilities for Open System Environment.

Language

English

Table of Content

Chapter 1. Introduction
Chapter 2. Implementing ESS Copy Services
Chapter 3. FlashCopy
Chapter 4. Peer-to-Peer Remote Copy (PPRC)
Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
Chapter 7. ESS Command Line Interface
Chapter 8. Open systems specifics
Chapter 9. Disaster Recovery
Chapter 10. IBM Systems Group Service Offerings
Appendix A. ESS Application Programming Interface (API)
Appendix B. Copy Services feature codes
Appendix C. Logical subsystems
Appendix D. System Adapter ID (SAID)
Appendix E. Sample naming convention
Appendix F. AIX and FlashCopy for backups
Appendix G. HACMP Remote Copy and ESS PPRC
ibm.com/redbooks
Front cover
IBM TotalStorage Enterprise Storage Server
Implementing ESS Copy
Services in Open Environments
Cathy Warrick
Paul Jenkin
Claus Schiefer
Jiri Spacek
Characteristics of the powerful ESS
Copy Services functions
FlashCopy and PPRC
fundamentals
Guidelines for ESS Copy
Services implementations


International Technical Support Organization
IBM TotalStorage Enterprise Storage Server
Implementing ESS Copy Services in Open
Environments
July 2004
SG24-5757-04

© Copyright International Business Machines Corporation 2001, 2003, 2004. All rights reserved.
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
Fifth Edition (July 2004)
This edition applies to the IBM TotalStorage Enterprise Storage Server Model 800 through ESS LIC 2.4.0.
Note: Before using this information and the product it supports, read the information in “Notices” on
page xxv.

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
iii
Contents
Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxv
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxvii
The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xxvii
Become a published author. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxix
Summary of changes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxi
July 2004, Fifth Edition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxi
Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 IBM TotalStorage Resiliency Family. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 ESS family of Enterprise Storage Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 IBM TotalStorage Enterprise Storage Servers Model 800 . . . . . . . . . . . . . . . . . . . . . . . 3
1.4.1 ESS Model 800 highlights. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 IBM TotalStorage Enterprise Storage Server Model 750 . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.1 Features of the ESS Model 750 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.2 Compatibility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 ESS Model 750 and 800 hardware features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.6.1 Host adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.7 Major components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.7.1 Extensive capacity and scalability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.8 ESS Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.8.1 Peer-to-Peer Remote Copy (PPRC). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.8.2 Peer-to-Peer Remote Copy Extended Distance (PPRC-XD) . . . . . . . . . . . . . . . . 12
1.8.3 FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.8.4 Incremental FlashCopy (V2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.8.5 Inband FlashCopy (V2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.8.6 Multiple Relationship FlashCopy (V2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.8.7 FlashCopy Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8.8 Data Set Level FlashCopy (V2 for OS/390® only) . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8.9 Asynchronous Cascading PPRC (V2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8.10 Asynchronous PPRC (V2). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8.11 PPRC over Fibre Channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8.12 ESS Application Programming Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8.13 Managing ESS Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.9 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.9.1 ESS Copy Services benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter 2. Implementing ESS Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 Copy Services terminology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.1 FlashCopy Version 1 (V1) and FlashCopy Version 2 (V2) . . . . . . . . . . . . . . . . . . 18
2.1.2 PPRC Version 1 (V1) and PPRC Version 2 (V2) . . . . . . . . . . . . . . . . . . . . . . . . . 18

iv
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.1.3 Copy Services Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.4 Active Copy Services server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.5 Copy Services client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.6 Primary Copy Services Server (LIC level prior to 2.2.x) . . . . . . . . . . . . . . . . . . . . 19
2.1.7 Backup Copy Services Server (LIC level prior to 2.2.x) . . . . . . . . . . . . . . . . . . . . 19
2.1.8 Copy Services server operating modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.9 Single-active configuration (LIC level prior to 2.2.x) . . . . . . . . . . . . . . . . . . . . . . . 20
2.1.10 Dual-active server configuration (LIC level 2.2.x or higher) . . . . . . . . . . . . . . . . 20
2.1.11 Mixed mode configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.1.12 The ESS Command Line Interface (CLI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 Network considerations for Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 ESS cluster communication and network configuration . . . . . . . . . . . . . . . . . . . . 21
2.2.2 Management via the WUI and network considerations. . . . . . . . . . . . . . . . . . . . . 21
2.2.3 Management using the CLI and network considerations . . . . . . . . . . . . . . . . . . . 22
2.3 ESS Copy Services (LIC level prior to 2.2.x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 Overview and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.2 Defining the Copy Services Domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.3 Planning for a disaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 ESS Copy Services (LIC level 2.2.x or higher). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.1 Overview and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.4.2 Defining the Copy Services Domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5 Copy Services Domain configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.1 Dual-active configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.2 Mixed configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5.3 Compatibility matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5.4 Task management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.5.5 Inband management considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.5.6 Migration from pre-LIC 2.2.x to LIC 2.2.x or higher. . . . . . . . . . . . . . . . . . . . . . . . 34
Chapter 3. FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.2 FlashCopy essentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 FlashCopy V2 enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.1 FlashCopy across different logical subsystems (LSS) . . . . . . . . . . . . . . . . . . . . . 37
3.3.2 Multiple Relationship FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.3 Incremental FlashCopy and the Reverse Restore option. . . . . . . . . . . . . . . . . . . 38
3.3.4 FlashCopy Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.3.5 Inband commands over the PPRC link. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.3.6 Establish time improvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4 FlashCopy tasks and options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.1 FlashCopy V1 establishment options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.4.2 FlashCopy V1 withdraw options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.4.3 FlashCopy V2 establishment options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4.4 FlashCopy V2 withdraw options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.4.5 Consistency Created task with FlashCopy V2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.4.6 FlashCopy V2 enhancements (LIC 2.4.x). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.5 Planning for FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.5.1 Hardware and software requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.5.2 Configuration planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.5.3 Resource planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.5.4 Data consistency considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.5.5 Test plan and Disaster Recovery plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.6 Operational considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Contents
v
3.6.1 Monitoring and managing FlashCopy pairs and volumes. . . . . . . . . . . . . . . . . . . 48
3.6.2 Using a FlashCopy target volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.6.3 Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.7 Performance considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.7.1 Placement of source and target volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.7.2 No background copy option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.7.3 Number of simultaneous FlashCopy pairs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.8 Practical examples using FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.8.1 Moving and migrating data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.8.2 Moving workload. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.8.3 Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.8.4 Application testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.8.5 Other examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Chapter 4. Peer-to-Peer Remote Copy (PPRC). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1 PPRC overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.2 Synchronous PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2.1 Synchronous PPRC initial copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2.2 Synchronous PPRC continuous operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.2.3 Synchronous PPRC volume states. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.2.4 Synchronous PPRC tasks and options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.3 PPRC Extended Distance (PPRC-XD). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.3.1 PPRC-XD operation and volume states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.3.2 PPRC-XD tasks and options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.3.3 Creating a consistent point-in-time copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.4 Synchronous PPRC and PPRC-XD comparison . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.4 Asynchronous Cascading PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4.1 Asynchronous Cascading PPRC operation and volume states . . . . . . . . . . . . . . 67
4.4.2 Asynchronous Cascading PPRC tasks and options. . . . . . . . . . . . . . . . . . . . . . . 69
4.4.3 Asynchronous Cascading PPRC mode combinations . . . . . . . . . . . . . . . . . . . . . 70
4.4.4 Creating a consistent copy with PPRC-XD as a remote pair . . . . . . . . . . . . . . . . 71
4.4.5 Outage scenarios with Asynchronous Cascading PPRC . . . . . . . . . . . . . . . . . . . 73
4.4.6 PPRC Failover and Failback with Asynchronous Cascading PPRC. . . . . . . . . . . 74
4.5 Asynchronous PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.5.1 Asynchronous PPRC overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.5.2 Asynchronous PPRC terminology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.5.3 Asynchronous PPRC operation and volume states . . . . . . . . . . . . . . . . . . . . . . . 75
4.5.4 Asynchronous PPRC tasks and options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.5.5 Setting up Asynchronous PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.5.6 Asynchronous PPRC management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.5.7 FlashCopy options for Asynchronous PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.6 PPRC data consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.6.1 Consistency types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.6.2 PPRC Consistency Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.7 Planning for PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.8 Choosing the best method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.8.1 Comparing PPRC features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.9 PPRC connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.9.1 ESCON links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.9.2 Configuring PPRC links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.9.3 PPRC paths definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.9.4 PPRC path failure alerts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.9.5 Performance considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

vi
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.10 PPRC using Fibre Channel links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.10.1 Configuration guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.10.2 Distance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.10.3 SAN fabric and networking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.10.4 WWNN and WWPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.10.5 Recognizing the ESS ports within the SAN fabric. . . . . . . . . . . . . . . . . . . . . . . 118
4.11 Asynchronous PPRC Bandwidth Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.12 Practical examples of PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.12.1 Site migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.12.2 Synchronous PPRC using static volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.12.3 Database log transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.12.4 Off-site backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0 . . . . . . . . . . . . 123
5.1 LIC level considerations within this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.2 Overview and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.3 Using a browser to access the ESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.3.1 Failure to connect to ESS Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.3.2 Location of ESS Copy Services server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.3.3 Restarting ESS Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.3.4 How to switch to the Backup server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.3.5 Steps following a disaster at the production site. . . . . . . . . . . . . . . . . . . . . . . . . 130
5.4 Volumes panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.4.1 Volume Information Panel button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5.4.2 Finding volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.4.3 Filtering volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.4.4 Multiple Selection Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5.5 Logical Subsystems panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.5.1 LSS Information Panel button. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5.5.2 Finding LSSs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5.5.3 Filtering LSSs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.5.4 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.6 The Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.6.1 Display Direct Connection Paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.6.2 Path Information Panel button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.7 Tasks panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5.7.1 Grouping and ungrouping tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.7.2 Removing a task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5.7.3 Running a task and viewing an error about a failed task . . . . . . . . . . . . . . . . . . 146
5.7.4 Modifying a task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.8 Administration panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.8.1 Working with the Copy Services logs and reports. . . . . . . . . . . . . . . . . . . . . . . . 148
5.8.2 Refreshing path and volume information for an ESS . . . . . . . . . . . . . . . . . . . . . 149
5.8.3 Managing the CLI user ID and password for an open-systems host . . . . . . . . . 149
5.9 Exiting ESS Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.10 Performing FlashCopy with the Web User Interface. . . . . . . . . . . . . . . . . . . . . . . . . 150
5.10.1 Establishing a FlashCopy pair. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
5.10.2 Getting Information about a FlashCopy pair. . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5.10.3 Withdrawing a FlashCopy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
5.10.4 FlashCopy Start Background Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
5.11 Performing PPRC with ESS Copy Services Web User Interface . . . . . . . . . . . . . . . 161
5.11.1 Establishing paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
5.11.2 Removing paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Contents
vii
5.11.3 Establishing a Synchronous PPRC pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.11.4 Establishing a PPRC Extended Distance (PPRC-XD) copy pair . . . . . . . . . . . 173
5.11.5 Converting a PPRC-XD copy pair to Synchronous PPRC . . . . . . . . . . . . . . . . 174
5.11.6 Creating a backup copy of volumes via PPRC Extended Distance . . . . . . . . . 176
5.11.7 Suspending a PPRC pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
5.11.8 Terminating a PPRC copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
5.11.9 Resynchronizing PPRC copy pairs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
5.11.10 Creating a PPRC Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.11.11 Freezing a PPRC Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.11.12 Thawing a PPRC Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
5.11.13 PPRC Failover and Failback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later . . . . . . . . . . 189
6.1 Overview and requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
6.2 ESS Copy Services WUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
6.3 Web Copy Services Domain: Tools and configuration . . . . . . . . . . . . . . . . . . . . . . . . 191
6.3.1 Copy Services Tools main menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
6.3.2 Defining and restarting the Web Copy Services Domain . . . . . . . . . . . . . . . . . . 192
6.4 Connecting to the Copy Services Web User Interface . . . . . . . . . . . . . . . . . . . . . . . . 196
6.4.1 Failure to connect to the ESS Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . 198
6.4.2 Restart the ESS Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
6.5 Volumes panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
6.5.1 Volumes panel icons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
6.5.2 Volume Information Panel button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
6.5.3 Finding volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
6.5.4 Filtering volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
6.5.5 Multiple Selection Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
6.6 Logical Subsystems panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
6.6.1 LSS Information Panel button. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
6.6.2 Finding LSSs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
6.6.3 Filtering LSSs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
6.6.4 LSS properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
6.7 The Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
6.7.1 Icons and symbols in the Paths panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
6.7.2 ESCON System Adapter ID (SAID) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
6.7.3 FCP System Adapter ID (SAID) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
6.7.4 Display Connection Paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
6.7.5 Path Information Panel button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
6.8 Tasks panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
6.8.1 Display task information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6.8.2 Grouping and ungrouping tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
6.8.3 Removing a task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.8.4 Running a task and viewing an error about a failed task . . . . . . . . . . . . . . . . . . 221
6.8.5 Modifying a task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.9 Administration panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
6.9.1 Working with the Copy Services logs and reports. . . . . . . . . . . . . . . . . . . . . . . . 222
6.9.2 Refreshing path and volume information on an ESS . . . . . . . . . . . . . . . . . . . . . 224
6.9.3 Managing the CLI user ID and password for an open systems host. . . . . . . . . . 224
6.10 Exiting ESS Copy services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
6.11 Performing FlashCopy operations with the WUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
6.11.1 Establishing a FlashCopy pair. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
6.11.2 Getting information about a FlashCopy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
6.11.3 Withdrawing a FlashCopy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

viii
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.11.4 Multiple FlashCopy using a single source volume . . . . . . . . . . . . . . . . . . . . . . 237
6.11.5 FlashCopy Start Background Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
6.11.6 Incremental FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
6.11.7 Reversing a FlashCopy relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6.11.8 Creating a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6.11.9 Freezing a FlashCopy Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6.11.10 Thawing a FlashCopy Consistency Group . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6.11.11 Inband FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
6.12 PPRC setup examples using the Web User Interface . . . . . . . . . . . . . . . . . . . . . . . 259
6.13 Paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
6.13.1 Establishing paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
6.13.2 Removing paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
6.14 Establishing a Synchronous PPRC pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.14.1 Options for Synchronous PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.15 PPRC-XD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
6.15.1 Establish a PPRC Extended Distance copy pair. . . . . . . . . . . . . . . . . . . . . . . . 271
6.15.2 Copy options for PPRC-XD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
6.15.3 Converting a PPRC-XD copy pair to Synchronous PPRC . . . . . . . . . . . . . . . . 275
6.15.4 Creating a backup copy of volumes via PPRC Extended Distance . . . . . . . . . 278
6.15.5 Suspending a PPRC pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
6.15.6 Terminating a PPRC copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
6.15.7 Resynchronizing PPRC copy pairs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
6.15.8 Creating a PPRC Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
6.15.9 Freezing a PPRC Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
6.15.10 Thawing a PPRC Consistency Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
6.15.11 Performing a PPRC Failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
6.15.12 Performing a PPRC Failback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
6.16 Establishing an Asynchronous Cascading PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . 300
6.16.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
6.16.2 Necessary tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
6.16.3 Allowed configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
6.16.4 Establishment order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
6.16.5 Options to make a volume eligible for cascading . . . . . . . . . . . . . . . . . . . . . . . 302
6.16.6 Establishing an Asynchronous Cascading PPRC with the WUI . . . . . . . . . . . . 302
6.16.7 PPRC Failover/Failback with Asynchronous Cascading PPRC . . . . . . . . . . . . 311
6.17 Managing Asynchronous PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
6.17.1 Before you start. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
6.17.2 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
6.17.3 Setting up Asynchronous PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
6.17.4 Starting Asynchronous PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
6.17.5 Pausing and terminating Asynchronous PPRC. . . . . . . . . . . . . . . . . . . . . . . . . 349
6.17.6 Switching sites after a disaster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
6.17.7 Remote site recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
6.17.8 Planned copy at remote site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
6.17.9 How to go back to the production site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
6.18 Summary of the benefits of the WUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
Chapter 7. ESS Command Line Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
7.1 ESS Copy Services CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
7.1.1 Requirements for Copy Services CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
7.1.2 Installing Copy Services CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
7.1.3 General command guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
7.1.4 Copy Services CLI Command description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

Contents
ix
7.2 ESS Storage Management CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
7.2.1 Requirements for the Storage Management CLI . . . . . . . . . . . . . . . . . . . . . . . . 394
7.2.2 Installing the Storage Management CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
7.2.3 General command guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
7.2.4 Copy Services related commands description . . . . . . . . . . . . . . . . . . . . . . . . . . 395
7.2.5 Other useful commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
Chapter 8. Open systems specifics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
8.1 AIX specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
8.1.1 AIX and FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
8.1.2 AIX and PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
8.1.3 Scripting Copy Services tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
8.2 Windows NT specifics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
8.2.1 Windows NT basic volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
8.2.2 Copy Services with Windows NT volume sets . . . . . . . . . . . . . . . . . . . . . . . . . . 420
8.3 Windows 2000 and Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
8.3.1 Copy Services with Windows 2000 volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
8.3.2 PPRC and Windows 2000 spanned volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . 426
8.3.3 ESSSync utility: Available by RPQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
8.3.4 Relating ESS volumes to Windows drive letters. . . . . . . . . . . . . . . . . . . . . . . . . 427
8.4 SUN Solaris and Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
8.4.1 Copy Services without a volume manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
8.4.2 Copy Services with a VERITAS Volume Manager . . . . . . . . . . . . . . . . . . . . . . . 432
8.5 HP-UX and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
8.5.1 HP-UX with FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
8.5.2 HP-UX with PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
8.6 OpenVMS with Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
8.6.1 OpenVMS with FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
8.6.2 OpenVMS and PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
8.7 Tru64 and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
8.7.1 Tru64 and FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
8.7.2 Tru64 and PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448
8.7.3 Managing Tru64 Persistent Reserve. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
Chapter 9. Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
9.1 Disaster recovery concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
9.1.1 What is Disaster Recovery? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
9.1.2 Business objectives of Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
9.1.3 Tiers of Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
9.2 Data consistency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
9.3 Types of PPRC Failover and Failback procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . 460
9.4 Copy Services server failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
9.4.1 Dual-active configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
9.4.2 Mixed configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
9.4.3 Single-active configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
9.5 Outage scenarios with Synchronous PPRC and PPRC-XD . . . . . . . . . . . . . . . . . . . . 462
9.5.1 Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
9.5.2 Planned failover without PPRC Failover/Failback options . . . . . . . . . . . . . . . . . 464
9.5.3 Failback after planned failover without PPRC Failover/Failback options . . . . . . 467
9.5.4 Planned failover with PPRC Failover/Failback options. . . . . . . . . . . . . . . . . . . . 469
9.5.5 Failback after planned failover with PPRC Failover/Failback options. . . . . . . . . 472
9.5.6 Unplanned failover without PPRC Failover/Failback options . . . . . . . . . . . . . . . 478
9.5.7 Failback after unplanned failover without PPRC Failover/Failback options . . . . 479

x
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
9.5.8 Unplanned failover with PPRC Failover/Failback options. . . . . . . . . . . . . . . . . . 480
9.5.9 Failback after unplanned failover with PPRC Failover/Failback options. . . . . . . 482
9.6 Failover/Failback with Asynchronous Cascading PPRC. . . . . . . . . . . . . . . . . . . . . . . 483
9.6.1 Switching to site B. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
9.6.2 Switching back to site A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486
9.7 Asynchronous PPRC outage scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
9.7.1 Asynchronous PPRC planned outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
9.7.2 Asynchronous PPRC unplanned outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
9.7.3 Evaluating the FlashCopy status query results. . . . . . . . . . . . . . . . . . . . . . . . . . 495
9.7.4 Creating a consistent copy of data with Asynchronous PPRC . . . . . . . . . . . . . . 496
9.8 Good practice guidelines for Disaster Recovery planning. . . . . . . . . . . . . . . . . . . . . . 498
9.8.1 CLI scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498
9.8.2 Script modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501
9.9 Multiple Device Manager Replication Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
9.10 Asynchronous PPRC Utilities for Open System Environment. . . . . . . . . . . . . . . . . . 507
9.11 Enterprise Remote Copy Management Facility (eRCMF). . . . . . . . . . . . . . . . . . . . . 508
Chapter 10. IBM Systems Group Service Offerings. . . . . . . . . . . . . . . . . . . . . . . . . . . 509
10.1 GDS for MSCS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
10.1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511
10.1.2 GDS Demo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512
10.1.3 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
10.2 IBM TotalStorage Data Manager for SQL 2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
10.2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
10.2.2 Benefits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
10.2.3 Hardware and software requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
10.2.4 Installation of TSDM for SQL 2000. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
10.2.5 TSDM for SQL 2000 GUI: Back up/restore a SQL Server 2000 database. . . . 525
10.2.6 TSDM for SQL 2000 command: Back up/restore SQL Server 2000 database. 528
10.2.7 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
10.3 Other Service Offerings for Windows and the ESS. . . . . . . . . . . . . . . . . . . . . . . . . . 530
10.3.1 IBM TotalStorage ESS - Exchange Integration Resource Pack. . . . . . . . . . . . 530
10.3.2 IBM TotalStorage ESSSync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
10.3.3 IBM TotalStorage Volume Set Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531
10.3.4 ESS API support for Microsoft Volume Shadow Copy Service (VSS) . . . . . . . 531
Appendix A. ESS Application Programming Interface (API). . . . . . . . . . . . . . . . . . . . 533
The ESS Application Programming Interface (API). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
ESS API characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534
ESS API components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
The ESS API interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
The client application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
Installing the ESS API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
Prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
Appendix B. Copy Services feature codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
Advanced functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542
Feature codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
ESS Model Exx and ESS Model Fxx. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
ESS Model 800 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544
PPRC over Fibre Channel links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546
Asynchronous PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546

Contents
xi
ESS Model 750 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546
Appendix C. Logical subsystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549
Device adapters and logical subsystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550
Logical subsystem (LSS). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
Host mapping to logical subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552
Fixed block considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552
Appendix D. System Adapter ID (SAID) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
SAID definition with ESCON links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
PPRC paths using ESCON links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
FCP System Adapter ID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
Appendix E. Sample naming convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561
Task names convention considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
Task name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
Task description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
Naming convention example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
PPRC operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563
FlashCopy operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564
Path operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565
LSS operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
Task Created using multiple volume selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566
Group operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
Operation codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
Appendix F. AIX and FlashCopy for backups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
Definition of a typical backup solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
Preparation after the first FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 572
Further FlashCopy invocations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
FlashCopy invocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
Backup scripts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576
Appendix G. HACMP Remote Copy and ESS PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . 583
HACMP Remote Copy and PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
PPRC mirroring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
HACMP Remote Copy integration with PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584
HACMP Remote Copy in an HACMP Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
Resource groups that include PPRC replicated resources. . . . . . . . . . . . . . . . . . . . . . 585
HACMP sites for HACMP Remote Copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
Primary and secondary volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
Fallover and fallback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
Sample configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 589
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590
How to get IBM Redbooks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593

xii
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
xiii
Figures
1-1 ESS family. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1-2 IBM’s Seascape architecture - ESS Model 800. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1-3 ESS high level design overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1-4 ESS Copy Services functions overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1-5 Allowable combinations of copy operation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2-1 Example of a domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2-2 Example of a domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2-3 Example of a domain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3-1 FlashCopy concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3-2 Incremental FlashCopy concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3-3 Incremental FlashCopy example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3-4 Consistency Groups with FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3-5 FlashCopy to a PPRC primary volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3-6 Database Backup scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3-7 Moving cluster environment using FlashCopy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4-1 PPRC initial volume synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4-2 PPRC write I/O cycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4-3 PPRC volume states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4-4 PPRC Extended Distance duplex pending XD volume state. . . . . . . . . . . . . . . . . . . 61
4-5 PPRC-XD volume state transitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4-6 Create a PPRC-XD consistent copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4-7 PPRC Extended Distance positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4-8 Volume pairs in an Asynchronous Cascading PPRC relationship. . . . . . . . . . . . . . . 68
4-9 Asynchronous Cascading PPRC intermediate volume states. . . . . . . . . . . . . . . . . . 68
4-10 Asynchronous Cascading PPRC intermediate volume information. . . . . . . . . . . . . . 69
4-11 Asynchronous Cascading PPRC with two sites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4-12 Create consistent copy with Asynchronous Cascading PPRC . . . . . . . . . . . . . . . . . 72
4-13 PPRC-XD environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4-14 Asynchronous PPRC processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4-15 Asynchronous PPRC Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4-16 Asynchronous PPRC setup steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4-17 Error sequence on Consistency Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4-18 Consistency group summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4-19 Freeze and resume summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4-20 ESS Copy Services configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4-21 PPRC comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4-22 PPRC ESCON connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4-23 Point-to-Point PPRC configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4-24 Configuration with one ESCON director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4-25 Configuration with two ESCON directors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4-26 Configuration using IBM 2029 Fibre Saver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4-27 Logical paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4-28 Configuring for availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4-29 Direct PPRC FCP links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4-30 Typical open systems host and PPRC connectivity using Fibre Channel . . . . . . . . 113
4-31 Logical paths between servers and LSSs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4-32 Logical paths for PPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4-33 Up to eight paths per LSS-LSS relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

xiv
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4-34 Logical path/physical path limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4-35 Fibre Channel distances without extending devices . . . . . . . . . . . . . . . . . . . . . . . . 116
4-36 ESS WWNN information in the ESS Specialist Welcome panel . . . . . . . . . . . . . . . 117
4-37 WWPN of ESS Fibre Channel Port 4-4-A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4-38 Determining the WWPN of an FCP port in the ESS . . . . . . . . . . . . . . . . . . . . . . . . 119
4-39 Site migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4-40 Database log transmission. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4-41 Split mirror implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5-1 ESS launch panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5-2 Copy Services start message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5-3 Main menu of ESS Copy Services Web User Interface. . . . . . . . . . . . . . . . . . . . . . 127
5-4 Configuration table of ESS Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5-5 The Tools panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5-6 Configuration table of ESS Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5-7 Volumes panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5-8 Volume Information Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5-9 Find Volume panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5-10 Filter volumes panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5-11 Logical Subsystem panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5-12 LSS Information panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5-13 Find LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5-14 Filtering logical subsystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5-15 LSS properties panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5-16 The Paths panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
5-17 SAID numbers of the ESS ESCON adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5-18 Example of the Paths panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5-19 Path Information panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
5-20 ESCON adapter without, with defined path, and failed establish path. . . . . . . . . . . 144
5-21 The Tasks panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5-22 Task information panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5-23 Administration panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
5-24 Volumes panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5-25 Task Wizard panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5-26 Select copy options panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5-27 Define Task panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5-28 FlashCopy volume display. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5-29 Information panel of a FlashCopy source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
5-30 Volumes panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
5-31 Withdraw a FlashCopy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
5-32 Define Task panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
5-33 Task Wizard: Withdraw FlashCopy pair. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
5-34 Task Wizard: Withdraw options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
5-35 Volumes panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
5-36 Task Wizard: Withdraw FlashCopy pair. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5-37 Task Wizard: Withdraw options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5-38 Paths panel of the Copy Services Web User Interface . . . . . . . . . . . . . . . . . . . . . . 162
5-39 Establish PPRC path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5-40 Task Wizard: Establish Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5-41 Task Wizard: Select outgoing ports panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
5-42 Establish PPRC options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
5-43 Path successfully established . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
5-44 Paths panel: Remove paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
5-45 Task Wizard: Remove paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Figures
xv
5-46 Task Wizard: Remove tasks options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5-47 Volume panel: establish PPRC pair. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
5-48 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
5-49 PPRC copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
5-50 Task Wizard: Establish PPRC path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
5-51 PPRC relationship in progress. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
5-52 PPRC pair in sync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
5-53 Information panel of PPRC source volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
5-54 Volumes panel: Converting PPRC - XD to Synchronous PPRC . . . . . . . . . . . . . . . 174
5-55 Task Wizard: Convert PPRC-XD to Synchronous PPRC . . . . . . . . . . . . . . . . . . . . 175
5-56 Task Wizard: Establish PPRC pair copy options. . . . . . . . . . . . . . . . . . . . . . . . . . . 175
5-57 Task Wizard: Suspend PPRC copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
5-58 Task Wizard: Specify logical subsystem to execute task panel. . . . . . . . . . . . . . . . 177
5-59 Task Wizard: Terminate PPRC copy pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
5-60 Task Wizard: Establish Synchronous PPRC copy pair . . . . . . . . . . . . . . . . . . . . . . 179
5-61 Task Wizard: Establish Synchronous PPRC copy pair options. . . . . . . . . . . . . . . . 179
5-62 Task Wizard: Establish paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5-63 Task Wizard: Select path options panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
5-64 Task Wizard: Define Task panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
5-65 Logical Subsystems panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5-66 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5-67 Task Wizard: Establish PPRC pair copy options. . . . . . . . . . . . . . . . . . . . . . . . . . . 185
5-68 Task Wizard: Establish PPRC pair copy options. . . . . . . . . . . . . . . . . . . . . . . . . . . 186
6-1 ESS welcome panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
6-2 Copy Services Tools panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
6-3 Copy Services action buttons from the Available Actions panel . . . . . . . . . . . . . . . 193
6-4 Define Copy Services Active Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
6-5 Defining the clients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
6-6 Define clients button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
6-7 Start CopyServices panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
6-8 Connecting to the Copy Services server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
6-9 Copy Services GUI: Welcome panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
6-10 Volume panel with source and target selected . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
6-11 Example of results of the rsPrimeServer command. . . . . . . . . . . . . . . . . . . . . . . . . 201
6-12 Volume Information Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
6-13 Find volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
6-14 Filter volumes panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
6-15 Multiple Selection Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
6-16 Logical Subsystem panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
6-17 LSS Information Panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
6-18 Find Storage Server LSS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
6-19 Filtering LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
6-20 LSS properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
6-21 View FCP paths status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
6-22 View ESCON paths status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
6-23 ESCON link: No path defined, path defined, and path definition failed . . . . . . . . . . 215
6-24 SAID numbers of the ESCON adapters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
6-25 SAID numbers of the FCP adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
6-26 Path Information Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
6-27 Tasks panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6-28 Task Information panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
6-29 Tasks panel: Grouping tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
6-30 Information panel: Task group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

xvi
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6-31 Information panel: Failing task report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6-32 Copy Services Administration panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
6-33 Copy Services Administration panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
6-34 Volumes panel with source and target volumes selected . . . . . . . . . . . . . . . . . . . . 226
6-35 Task Wizard panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
6-36 Define task panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
6-37 Define task panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
6-38 FlashCopy lightning bolts icons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
6-39 Volume Information Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
6-40 Volumes panel: Select volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
6-41 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
6-42 Task Wizard: Select withdraw options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
6-43 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
6-44 Volumes panel: Selecting a volume as source and target for FlashCopy withdraw 235
6-45 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6-46 Task Wizard: Select withdraw options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6-47 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
6-48 FlashCopy source and Multiple Relationship FlashCopy icons. . . . . . . . . . . . . . . . 237
6-49 Volumes panel: The target volume selected as source and target . . . . . . . . . . . . . 238
6-50 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
6-51 Task Wizard: Select withdraw options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
6-52 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
6-53 Volumes panel: Select volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6-54 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
6-55 Task panel: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
6-56 Volumes panel: Result of a Start Recording FlashCopy . . . . . . . . . . . . . . . . . . . . . 243
6-57 Information panel: Source volume and start recording . . . . . . . . . . . . . . . . . . . . . . 244
6-58 Volumes panel: Selecting volume for an Incremental FlashCopy . . . . . . . . . . . . . . 245
6-59 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6-60 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
6-61 Change Recording enabled: Background copy finished . . . . . . . . . . . . . . . . . . . . . 247
6-62 Volumes panel: Selecting volume for a Reverse Restore FlashCopy. . . . . . . . . . . 248
6-63 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
6-64 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6-65 Fast Reverse Restore options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
6-66 Fast Reverse Restore in progress. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6-67 Volumes panel: Selecting volumes using Multiple Selection Mode. . . . . . . . . . . . . 253
6-68 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
6-69 Task Wizard: Select copy options panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
6-70 Task Wizard: Define Task panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
6-71 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6-72 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
6-73 Task Window: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
6-74 Volumes panel: Volume selection for a FlashCopy with Inband command option . 257
6-75 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
6-76 Task Wizard: Choose LSS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
6-77 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
6-78 Volumes panel: Result of a FlashCopy with Inband Command option . . . . . . . . . . 259
6-79 Establishing paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
6-80 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
6-81 Select outgoing port: ESCON link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
6-82 Select outgoing port: FCP link. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
6-83 Select path options: ESCON links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

Figures
xvii
6-84 Select path options: FCP links. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6-85 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
6-86 ESCON path successfully established. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
6-87 FCP path successfully established . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
6-88 Selecting an established path to be removed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6-89 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6-90 Task Wizard: Select path options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6-91 Volumes panel: Selecting source and target volumes for PPRC. . . . . . . . . . . . . . . 267
6-92 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
6-93 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6-94 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
6-95 Two volumes in a PPRC relationship and in copy pending status. . . . . . . . . . . . . . 270
6-96 Information panel on a PPRC volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
6-97 PPRC volume in full duplex mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
6-98 Establishing a PPRC-XD between two volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
6-99 Task Wizard: Select copy option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
6-100 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
6-101 Volumes panel: PPRC-XD established between volumes 700 and 605 . . . . . . . . . 274
6-102 Volumes panel and Task Wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
6-103 Task Wizard: Select copy options for PPRC-XD. . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6-104 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6-105 Volumes panel: Resynchronizing volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
6-106 Volumes panel: Volumes are in suspended state after a fullcopy is established. . . 277
6-107 Task Wizard: Selecting Suspend PPRC copy pair option . . . . . . . . . . . . . . . . . . . . 279
6-108 Task Wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
6-109 Volumes panel: Result of suspend task with Schedule with the source option. . . . 280
6-110 Volumes panel: Result of suspend task with Schedule with the target option. . . . . 280
6-111 Volumes panel: Two volumes in a PPRC relationship selected . . . . . . . . . . . . . . . 281
6-112 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
6-113 Task Wizard: Specify logical subsystem to execute the task. . . . . . . . . . . . . . . . . . 282
6-114 Volumes panel: Result of a terminate PPRC with Schedule with target option. . . . 282
6-115 Volumes panel: Result of a terminate PPRC with Schedule with target option. . . . 283
6-116 Volumes panel and Task Wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
6-117 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
6-118 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
6-119 Paths panel: Establishing a path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
6-120 Task Wizard: Select path options - FCP link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
6-121 Task Wizard: Select path options - ESCON link . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
6-122 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
6-123 LSS panel: LSS source selected. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
6-124 LSS panel: LSS target selected. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
6-125 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
6-126 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
6-127 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
6-128 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
6-129 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
6-130 Volumes panel: Volume selection for a PPRC Failover. . . . . . . . . . . . . . . . . . . . . . 293
6-131 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
6-132 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
6-133 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
6-134 Volumes panel: Result of a failover task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
6-135 Volumes panel: Selecting the volume for a PPRC Failback task. . . . . . . . . . . . . . . 297
6-136 Task Wizard: Select task type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

xviii
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6-137 Task Wizard: Select copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
6-138 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
6-139 Volumes panel: Result of a failback task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
6-140 PPRC establish path: Local to intermediate site (ESS at LIC level 2.2.0). . . . . . . . 303
6-141 PPRC establish path: Intermediate to remote site (ESS at LIC level 2.2.0) . . . . . . 304
6-142 Establish PPRC-XD volume pair: Intermediate to remote site volume . . . . . . . . . . 305
6-143 Select copy option panel: Asynchronous Cascading PPRC . . . . . . . . . . . . . . . . . . 305
6-144 Task Wizard: Define Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
6-145 Volume Information Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
6-146 Intermediate volume as a PPRC primary and PPRC secondary. . . . . . . . . . . . . . . 308
6-147 Select copy option panel: Copy entire volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
6-148 Define Task Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
6-149 Asynchronous Cascading PPRC: Local pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
6-150 Asynchronous Cascading PPRC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
6-151 Asynchronous Cascading PPRC configuration used in this example . . . . . . . . . . . 312
6-152 Path successfully defined between site A and site B LSSs. . . . . . . . . . . . . . . . . . . 313
6-153 Path successfully defined between site B and site C LSSs. . . . . . . . . . . . . . . . . . . 313
6-154 Options to create the XD pair with. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
6-155 Remote PPRC-XD volume pair established . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
6-156 Options to create the local PPRC-SYNC pair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
6-157 Local PPRC-SYNC pair during initial synchronization (duplex pending). . . . . . . . . 316
6-158 Doing a PPRC Failover on site B volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
6-159 PPRC Failover to site B completed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
6-160 Information Panel for site B volume after failover . . . . . . . . . . . . . . . . . . . . . . . . . . 318
6-161 Display of remote PPRC-XD pair after suspending. . . . . . . . . . . . . . . . . . . . . . . . . 319
6-162 Site B to site A path established . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
6-163 PPRC Failback on site B volume: Task options. . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
6-164 Volumes in LSS17 & LSS16 on ESSA while PPRC Failback is proceeding . . . . . . 321
6-165 Volumes in LSS17 & LSS16 on ESSA after PPRC Failback on B has completed . 321
6-166 Information Panel for site B volume after PPRC Failback. . . . . . . . . . . . . . . . . . . . 322
6-167 Path successfully established from site A LSS16 to site B LSS17 . . . . . . . . . . . . . 323
6-168 Local PPRC volume pair: State before PPRC Failover. . . . . . . . . . . . . . . . . . . . . . 323
6-169 Local PPRC volume pair: State after PPRC Failover task on volume A . . . . . . . . . 324
6-170 PPRC Failback task: Copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
6-171 PPRC Failback task completed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
6-172 Information Panel for the site B volume after Failback to site A . . . . . . . . . . . . . . . 325
6-173 Remote volume pair before resynchronization (Trusted Primed for Resynch) . . . . 326
6-174 Re-synchronization between B and C volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
6-175 Information Panel for the site C volume after resynchronization. . . . . . . . . . . . . . . 327
6-176 Define Task wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
6-177 Configuration used for describing Asynchronous PPRC. . . . . . . . . . . . . . . . . . . . . 330
6-178 Establishing paths: Paths panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
6-179 Establishing Paths: Select primary LSS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
6-180 Establishing Paths: Work with Paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
6-181 Establishing Paths: Select secondary LSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
6-182 Establishing Paths: Selecting path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
6-183 Establishing Paths: Establish Paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
6-184 Establishing Paths: Outgoing ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
6-185 Establishing Paths: Select path options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
6-186 Establishing Paths: Display established paths. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
6-187 Creating PPRC-XD pairs: Select volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
6-188 Creating PPRC-XD pairs: Copy options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
6-189 Creating PPRC-XD pairs: View results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

Figures
xix
6-190 Creating FlashCopy pairs: Selecting volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
6-191 Creating FlashCopy pairs: Copy options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
6-192 Creating FlashCopy pairs: Sequence number. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
6-193 Creating FlashCopy pairs: Viewing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
6-194 Defining Asynchronous PPRC Session: Selecting LSS. . . . . . . . . . . . . . . . . . . . . . 341
6-195 Defining Asynchronous PPRC Session: Choosing task . . . . . . . . . . . . . . . . . . . . . 341
6-196 Defining Asynchronous PPRC session: Select copy options. . . . . . . . . . . . . . . . . . 342
6-197 Adding Volumes to session: Selecting volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
6-198 Adding Volumes to session: Selecting the session . . . . . . . . . . . . . . . . . . . . . . . . . 343
6-199 Adding Volumes to the session: Viewing the status . . . . . . . . . . . . . . . . . . . . . . . . 344
6-200 Starting Asynchronous PPRC session: Selecting the LSS . . . . . . . . . . . . . . . . . . . 345
6-201 Starting Asynchronous PPRC session: Selecting session and values . . . . . . . . . . 346
6-202 Starting Asynchronous PPRC session: Selecting Subordinates . . . . . . . . . . . . . . . 346
6-203 Starting Asynchronous PPRC session: Viewing results . . . . . . . . . . . . . . . . . . . . . 347
6-204 Starting Asynchronous PPRC session: Viewing volume status. . . . . . . . . . . . . . . . 348
6-205 Starting Asynchronous PPRC session: Viewing status panel . . . . . . . . . . . . . . . . . 349
6-206 Pausing session: Task selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
6-207 Pausing session: Selecting options and session. . . . . . . . . . . . . . . . . . . . . . . . . . . 350
6-208 Pausing session: Selecting Subordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
6-209 Pausing session: Viewing status panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
6-210 Failover to secondary: Selecting volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
6-211 Failover to secondary: Selecting options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
6-212 Failover to secondary: Viewing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
6-213 FlashCopy status: Not revertible . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
6-214 FlashCopy status: Revertible. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
6-215 Commit-Revert: Task selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
6-216 Commit-Revert: Options selection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
6-217 Commit-Revert: Sequence number. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
6-218 Fast Reverse Restore: Selecting volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
6-219 Fast Reverse Restore: Selecting options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
6-220 Fast Reverse Restore: Sequence number. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
6-221 Fast Reverse Restore: Viewing progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
6-222 Fast Reverse Restore: Viewing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
6-223 Fast Reverse Restore: Re-instating FlashCopy pairs . . . . . . . . . . . . . . . . . . . . . . . 363
6-224 FlashCopy to D: Select volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
6-225 FlashCopy to D: Select options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
6-226 FlashCopy to D: View results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
6-227 Failback to primary: Paths, selecting LSSs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
6-228 Failback to primary: Paths, selecting ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
6-229 Failback to primary: Selecting volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
6-230 Failback to primary: Selecting options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
6-231 Failback to primary: Viewing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
6-232 Failback to primary: Go to synchronous, selecting volumes . . . . . . . . . . . . . . . . . . 370
6-233 Failback to primary: Go to synchronous, selecting options . . . . . . . . . . . . . . . . . . . 371
6-234 Failback to primary: Go to synchronous, viewing results. . . . . . . . . . . . . . . . . . . . . 371
6-235 Failback to primary: Failover, selecting volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . 372
6-236 Failback to primary: Failover, selecting options. . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
6-237 Failback to primary: Failover, viewing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
6-238 Reinstating PPRC: Selecting volumes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
6-239 Reinstating PPRC: Selecting options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
6-240 Reinstating PPRC: Viewing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
8-1 lspv after pv=clear . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
8-2 Recreated FlashCopy target volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412

xx
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
8-3 Target file system stanza. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
8-4 PPRC phantom disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
8-5 Phantom hdisks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
8-6 Original time stamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
8-7 Updated source time stamp. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
8-8 Update secondary server’s ODM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
8-9 ESS logical volume serial numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
8-10 View of disks as seen by Disk Administrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
8-11 Output from rsList2105s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
8-12 View after rsPrimeServer has been run. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
8-13 Datapath query device command output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
8-14 rsList2105s.sh on SUN Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
8-15 Output from mount -v on Solaris . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
8-16 Import of a FlashCopy target. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
8-17 Output showing attached SCSI disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
8-18 Output showing attached FC disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
8-19 OpenVMS volumes with UDID labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
8-20 Established source and target. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
8-21 Checking consistency of FlashCopy target volume. . . . . . . . . . . . . . . . . . . . . . . . . 442
8-22 Status of FlashCopy mount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
8-23 Mount FlashCopy target for write. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
8-24 Change FlashCopy target volume label. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
8-25 Providing system-wide access to FlashCopy target . . . . . . . . . . . . . . . . . . . . . . . . 444
8-26 essvol script. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
8-27 Output of essvol script. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
8-28 Contents of AdvFS domain DOM1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
8-29 Manual AdvFS domain creation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
8-30 Identifying AdvFS domain ID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448
8-31 Checking for Persistent Reserve. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
8-32 Trying to remove the PR from another system in the key list . . . . . . . . . . . . . . . . . 450
8-33 Checking the key table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
8-34 Trying to remove the PR from a system not in the key list. . . . . . . . . . . . . . . . . . . . 451
9-1 Components of Disaster Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
9-2 Business objectives of Disaster Recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
9-3 Tiers of Disaster Recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456
9-4 Disaster recovery portfolio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
9-5 PPRC site switch preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
9-6 PPRC-XD site switch preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
9-7 Initiating reverse synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466
9-8 PPRC resumes at application site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
9-9 PPRC-XD resumes at application site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
9-10 Create failover task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
9-11 Task Wizard failover option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
9-12 Volume state after failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
9-13 Create failback task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
9-14 Volume state after the failback. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
9-15 Failover after recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
9-16 Combining the options PPRC Failover and PPRC Failback . . . . . . . . . . . . . . . . . . 478
9-17 PPRC Failover option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
9-18 PPRC Failback option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
9-19 Resume application site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
9-20 Volumes states after Failover command to site B volumes. . . . . . . . . . . . . . . . . . . 485
9-21 Swap I/O to site B volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

Figures
xxi
9-22 Volumes states after failback to site B volumes and quiesce application I/O . . . . . 487
9-23 Volumes states after PPRC Failover command to site A. . . . . . . . . . . . . . . . . . . . . 488
9-24 Volumes states after PPRC Failback command to site A . . . . . . . . . . . . . . . . . . . . 488
9-25 Return to full Asynchronous Cascading PPRC relationship . . . . . . . . . . . . . . . . . . 489
9-26 Asynchronous PPRC site switch preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
9-27 Asynchronous PPRC unplanned outage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492
9-28 Starting applications at the Recovery site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493
9-29 Switching back to the Application site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
9-30 Creating a consistent copy of data at the recovery site. . . . . . . . . . . . . . . . . . . . . . 497
10-1 GDS topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
10-2 GDS example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
10-3 All three groups are owned by RootBeer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
10-4 Quorum resource (Disk E:) is on RootBeer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
10-5 The disk manager of RootBeer sees every disk . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
10-6 PPRC primary and secondary role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
10-7 Initiate move group on RootBeer for all three groups . . . . . . . . . . . . . . . . . . . . . . . 515
10-8 All groups are now owned by MountainDew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
10-9 PPRC primary and secondary role exchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
10-10 The groups are now again owned by RootBeer. . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
10-11 PPRC role exchange again . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
10-12 Total network communication fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
10-13 Storage communication fault. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519
10-14 Server fault. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
10-15 ESS fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
10-16 Site fault. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522
10-17 All communication fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522
10-18 TSDMSql backup window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526
10-19 Finish window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
10-20 TSDMSQL restore view. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
10-21 Performance monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529
10-22 SQL log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530
A-1 Components of the API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535
A-2 API method of operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
A-3 ESS management interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
C-1 Logical subsystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550
C-2 Sample of logical subsystem configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551
D-1 Physical ESCON port location. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556
D-2 SAID numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557
D-3 Determine pathing to remote ESS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
D-4 System Adapter ID (SAID) identification with Fibre Channel ports . . . . . . . . . . . . . 559
E-1 Task Name and Description prompt window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 562
F-1 lspv of hdisk26. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573
F-2 lscfg on hdisk16. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
F-3 Identifying FlashCopy target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
G-1 Sample PPRC configuration with HACMP Remote Copy . . . . . . . . . . . . . . . . . . . . 587

xxii
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
xxiii
Tables
2-1 Example of configuration: dual-active servers and clients. . . . . . . . . . . . . . . . . . . . . 27
2-2 Example of mixed configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2-3 Example of configuration: dual-active servers and pre-LIC 2.2.x clients. . . . . . . . . . 32
2-4 Supported pairs of server in the Copy Services Domain. . . . . . . . . . . . . . . . . . . . . . 33
4-1 Comparison of PPRC and host-based mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4-2 Comparison of Synchronous PPRC and PPRC-XD characteristics . . . . . . . . . . . . . 66
4-3 Asynchronous Cascading PPRC mode combinations. . . . . . . . . . . . . . . . . . . . . . . . 70
4-4 Minimum requirements for PPRC features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4-5 Task options for establishing PPRC relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
4-6 Fibre Channel versus ESCON characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5-1 Meaning of icons on Volume panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5-2 Meaning of icons on Logical Subsystems panel . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5-3 Meaning of the Paths panel icons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6-1 Meaning of the icons and colors in the Volume panel. . . . . . . . . . . . . . . . . . . . . . . 201
6-2 Meaning of icons on Logical Subsystems panel . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
6-3 Meaning of the Paths panel icons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
6-4 Summary of the PPRC Failover creation and results . . . . . . . . . . . . . . . . . . . . . . . 296
6-5 Summary of the PPRC Failover and Failback creation and results. . . . . . . . . . . . . 300
6-6 Revertible status and actions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357
7-1 Platform specific notations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380
7-2 Path status table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
9-1 FlashCopy status query results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496
B-1 Copy Services Feature Codes V1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
B-2 Copy Services Feature Codes V2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
B-3 PPRC and FlashCopy V1 and V2 Feature Codes for the ESS Models Fxx . . . . . . 544
B-4 ESS Model 800 (2105-800) and IBM 2240 ESS function authorization features. . . 545
B-5 ESS Model 750 and IBM 2240 ESS Authorization features . . . . . . . . . . . . . . . . . . 547
E-1 Operation codes for PPRC and PPRC-XD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
E-2 Operation codes for FlashCopy operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
E-3 Operation code for path management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569
E-4 Operation codes for task defined at the LSS level. . . . . . . . . . . . . . . . . . . . . . . . . . 570
F-1 Contents of samplevg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573

xxiv
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
xxv
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions are
inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS
PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and
distribute these sample programs in any form without payment to IBM for the purposes of developing, using,
marketing, or distributing application programs conforming to IBM's application programming interfaces.

xxvi
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AIX 5L™
AIX®
Enterprise Storage Server®
ESCON®
Eserver®
Eserver®
FlashCopy®
FICON®
HACMP™
IBM®
ibm.com®
iSeries™
MVS™
NUMA-Q®
OS/390®
OS/400®
Parallel Sysplex®
pSeries®
Redbooks™
Redbooks (logo) ™
RS/6000®
S/390®
Seascape®
System/390®
Tivoli®
TotalStorage®
VSE/ESA™
Wave®
WebSphere®
xSeries®
z/OS®
z/VM®
zSeries®
The following terms are trademarks of other companies:
Intel, Intel Inside (logos), and Pentium are trademarks of Intel Corporation in the United States, other
countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.
Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun
Microsystems, Inc. in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product, and service names may be trademarks or service marks of others.

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
xxvii
Preface
This IBM® Redbook describes the copy functions available with the IBM TotalStorage
Enterprise Storage Server (ESS). The powerful ESS Copy Services functions are explained
in detail, and their respective characteristics are thoroughly covered. This redbook also gives
information about how to manage the various ESS Copy Services functions, and finally
discusses their implementations.
Because this redbook provides a broad understanding of the ESS Copy Services functions,
as well as going into detail about the management interfaces and the implementation
considerations, we recommend it for IT professionals who are planning the implementation of
any of the ESS Copy Services functions in an open-systems environment.
This fifth edition of the redbook has been updated with the latest ESS Copy Services
functions available with LIC level 2.4.0:
Brief overview of the ESS Model 750
ESS LIC 2.4.0 additional functions:
– Asynchronous PPRC.
– A FlashCopy® target can now be a PPRC primary.
Minor additions for ESS API support, Multiple Device Manager Replication Manager, and
the Asynchronous PPRC Utilities for Open System Environment.
Also, some Web User Interface panels have changed with the introduction of LIC 2.4.0 and
these changes are presented in this edition of the redbook.
The team that wrote this redbook
This redbook was produced by a team of specialists from around the world working at the
International Technical Support Organization, San Jose Center.
Cathy Warrick is a project leader and Certified IT Specialist in the IBM International
Technical Support Organization. She has over 25 years of experience in IBM with large
systems, open systems, and storage, including education on products internally and for the
field. Prior to joining the ITSO, she developed the Technical Leadership education program
for the IBM and Business Partner’s technical field force.
Paul Jenkin has been with IBM for 23 years, starting as a hardware field engineer and later
as a hardware specialist. He has worked on most of IBM's products over that period from PCs
to zSeries®, including most of IBM's DASD and storage products. Paul holds a NZ CIT
Diploma in Computer Engineering. He has managed large and complex customer sites,
including working alongside customers to manage and teach complex system recovery. In his
current role as ITS Technical Team Leader for New Zealand, Paul provides leadership and
specialist support to ITS Service Representatives across all platforms in both service and
implementation and also provides backup support to Australia. Specialist areas include
zSeries, all storage products disk and tape, SAN, networking, and complex problem
management.
Claus Schiefer is an IT Specialist and works for the Field Technical Sales Support in
Germany. He joined IBM in 1991and was a Systems Engineer for MVS™ and communication
systems during his first three years. He has 10 years of storage experience in the disk and

xxviii
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
tape storage systems field with a focus on z/OS®. His areas of expertise include high end
disk storage systems with PPRC, FlashCopy, XRC, and high end tape storage systems.
Jiri Spacek is an IT Consultant at GC System a.s., an IBM Business Partner in the Czech
Republic. He has seven years of experience in the IT field. His areas of expertise include IBM
^ pSeries® and AIX®, HACMP™, and disk storage subsystems. Currently he is an
IBM Certified Advanced Technical Expert for pSeries and AIX 5L™ and an IBM Certified
Specialist for Enterprise Disk Solutions.
Figure 1 Jiri, Paul, Cathy, and Claus
The authors of previous editions of this book are:
Barry Mellish Paresh Chudasma Susanne Lukas Massimo Rosichini
Andrew Beyer Rainer Wolafka Vladmir Blazek Cathy Warrick
Olivier Alluis Torsten Rothenwaldt Jiri Spacek Nancy Roper
Toga Gursu Carlin Smith Gustavo Castets George Kozakos
Donald (Chuck) Laing
Special thanks to Rosemary McCutchen and Craig Gordon from IBM Advanced Technical
Support, ATS Gaithersburg, for their support with the setup of the environment we used, as
well as their invaluable feedback when testing the new functions and discussing with us the
technical considerations for our redbook
Thanks to the following people for their contributions to this project:
John Amann
Bob Bartfai
Werner Bauer
Charlie Burger
Shelley Carton
Steve Chase
David Chou
Steven Cook
Richard Grossman
Matthias Gubitz
Joe Hyde
Lloyd Johnson
Amir Jafri
Frank Krueger
Troy Lee
Susanne Lukas
Lu Nguyen
Alison Pate

Preface
xxix
Don Resnick
Brian Sherman
Gail Spear
Warren Stanley
Tom Taleck
John Thompson
Jim Tuckwell
Ricardo Urbanejo
Glenn Wightwick
Steve Wilson
Olga Yiparaki
Many thanks to the editors:
Emma Jacobs and Wade Wallace
Become a published author
Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with
specific products or solutions, while getting hands-on experience with leading-edge
technologies. You'll team with IBM technical professionals, Business Partners and/or
customers.
Your efforts will help increase product acceptance and customer satisfaction. As a bonus,
you'll develop a network of contacts in IBM development labs, and increase your productivity
and marketability.
Find out more about the residency program, browse the residency index, and apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our Redbooks™ to be as helpful as possible. Send us your comments about this or
other Redbooks in one of the following ways:
Use the online Contact us review redbook form found at:
ibm.com/redbooks
Send your comments in an Internet note to:
redbook@us.ibm.com
Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. QXXE Building 80-E2
650 Harry Road
San Jose, California 95120-6099

xxx
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
xxxi
Summary of changes
This section describes the technical changes made in this edition of the book and in previous
editions. This edition may also include minor corrections and editorial changes that are not
identified.
Summary of Changes
for SG24-5757-04
for IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open
Environments
as created or updated on January 23, 2006.
July 2004, Fifth Edition
This revision reflects the addition, deletion, or modification of new and changed information
described below.
New information
Brief overview of the ESS Model 750
ESS LIC 2.4.0 additional functions:
– Asynchronous PPRC.
– A FlashCopy target can now be a PPRC primary.
Minor additions for ESS API support, Multiple Device Manager Replication Manager, and
the Asynchronous PPRC Utilities for Open System Environment
Changed information
Additional scenarios using Asynchronous PPRC
Updated Web User Interface screens and CLI commands
Updated on 08/19/2004 to fix an error in Chapter 6, replace images, 6-128, 6-219, 6-221, and
6-222 and add a notebox.
Updated on 01/18/2006 to fix an error in Chapter 6, p.247 and images, 6-53, 6-56, 6-58, 6-62,
and 6-63.

xxxii
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
1
Chapter 1.
Introduction
This introductory chapter reviews the topics covered in this book and introduces the IBM
TotalStorage Resiliency Family of products, which includes the IBM TotalStorage Enterprise
Storage Server (ESS) and ESS Copy Services, which will be discussed in-depth in this book.
The features of ESS Copy Services are summarized and the key components described.
We review the IBM Seascape® Architecture and the IBM TotalStorage Enterprise Server
(ESS). Also introduced are the models of the IBM TotalStorage Enterprise Storage Server,
including the Model 800 and the new Model 750. This is followed by a discussion on the
strategies for Disaster Recovery and where the IBM TotalStorage Enterprise Storage Server
(ESS) advanced copy functions are positioned.
When planning your Disaster Recovery solutions, the following redbook should be referenced
for further detailed discussion: IBM TotalStorage Solutions for Disaster Recovery,
SG24-6547.
We introduce the features of the ESS Copy Services functions and briefly discuss some of
the benefits.
1

2
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
1.1 Overview
As businesses become more and more dependent on information technology to conduct their
operations and stay competitive, the availability of their processing facilities becomes crucial.
Today, most businesses require a high level of availability, which extends to continuous
availability, 24 hours a day and seven days a week (24x7). A lengthy outage could lead to
significant financial losses, loss of credibility with customers, and maybe even a total failure of
business. Therefore, the ability to provide continuous availability for the major applications is
more often than not a necessity for business survival. A key component of a highly available
system is the storage subsystem. It is essential that data is available at all times and that
downtime for data backup and software maintenance is minimized or more preferably
eliminated.
These important demands on the storage system are fulfilled with the Enterprise Storage
Server® (ESS) and its Copy Services functions. The ESS Copy Services provide replication
of mission critical data, point-in-time and incremental FlashCopy, and dynamic synchronous
and asynchronous mirroring to a remote site.
1.2 IBM TotalStorage Resiliency Family
The IBM Total Storage Resiliency Family is comprised of IBM TotalStorage Resiliency Core
Technology, an extensive set of hardware and software features and products, IBM
TotalStorage Resiliency Automation, and integrated software and services packages. The
IBM TotalStorage Resiliency Family is designed to help you implement storage
infrastructures to help keep your business running 24 hours a day, 7 days a week.
The following ESS Model 750 and 800 functions are key components of the IBM TotalStorage
Resiliency Core Technology:
FlashCopy, also known as IBM TotalStorage FlashCopy
PPRC:
– Synchronous PPRC, also known as IBM TotalStorage Metro Mirror
– Asynchronous PPRC, also known as IBM TotalStorage Global Mirror
– Asynchronous Cascading PPRC, also known as IBM TotalStorage Metro/Global Copy
– PPRC Extended Distance, also known as IBM TotalStorage Global Copy
XRC (Extended Remote Copy)
- Model 800 only, z/OS only
:
– XRC, also known as IBM TotalStorage z/OS Global Mirror
– zSeries (three-site solution using Synchronous PPRC and XRC), also known as IBM
TotalStorage z/OS Metro/Global Mirror
1.3 ESS family of Enterprise Storage Servers
The ESS family of Enterprise Storage Servers was expanded in April 2004 with the
announcement of the ESS Model 750. The family now includes the ESS Model 800, ESS
Model 800 with Turbo option, and the new ESS Model 750. In Figure 1-1 on page 3 are some
general descriptions of each model and their design points.

Chapter 1. Introduction
3
Figure 1-1 ESS family
1.4 IBM TotalStorage Enterprise Storage Servers Model 800
The IBM TotalStorage Enterprise Storage Server (ESS) is IBM’s most powerful disk storage
server, developed using IBM Seascape architecture. The ESS provides un-matchable
functions for all the IBM ^ family of e-business servers, and also for the non-IBM (that
is, Intel®-based and UNIX®-based) families of servers. Across all of these environments, the
ESS features unique capabilities that allow it to meet the most demanding requirements of
performance, capacity, and data availability that the computing business may require.
The Seascape architecture is the key to the development of IBM’s storage products.
Seascape allows IBM to take the best of the technologies developed by the many IBM
laboratories and integrate them, producing flexible and upgradeable storage solutions. This
Seascape architecture design has allowed the IBM TotalStorage Enterprise Storage Server
to evolve from the initial E models to the succeeding F models, to the current 750 and 800
models, each featuring new, more powerful hardware and functional enhancements, and
always integrated under the same successful architecture with which the ESS was originally
conceived.
The move to on demand business presents companies with both extraordinary opportunities
and significant challenges. Consequently, companies also face an increase in critical
requirements for more information that is universally available online, around the clock, every
day of the year.
To meet the requirements of on demand business, where massive swings in the demands
placed on your systems are common, and continuous operation is imperative, you need very
high-performance and intelligent storage technologies and systems, which can support any
server application in your business, today and into the future. The IBM TotalStorage
IBM TotalStorage
®
ESS Family of Enterprise Storage Servers
 ESS Model 800

Excellent capacity
scalability supporting up to
55.9 TB

High performance
scalability designed to
meet/exceed the vast
majority of customer
workload requirements

Wide array of disk
capacities and speeds
 ESS Model 750

For moderate capacity needs -
supports up to 4.6 TB

Designed to meet lower tier
customer workload
requirements

Designed to offer attractive
pricing for modest capacity
requirements

Designed to offer non-disruptive
path to ESS Model 800
 ESS Model 800
with Turbo option

Up to 30% greater
throughput than ESS
base model 800

Designed to meet the
needs of very high
performance
environments

Designed to offer non-
disruptive Field
upgradeability from
ESS Model 800 Base
Designed to avoid single points of
Failure/Repair
Designed to support 24X7 operations
Offers flexibility in efficient
management options
Wide array of heterogeneous host
attachments
20+ autonomic functions
Rich set of optional advanced
functions
Designed to offer low total cost of
ownership
Included in all ESS Models

4
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Enterprise Storage Server has set new standards in function, performance, and scalability in
these most challenging environments.
1.4.1 ESS Model 800 highlights
Figure 1-2 gives an overview of the Seascape architecture on the ESS Model 800.
Figure 1-2 IBM’s Seascape architecture - ESS Model 800
Since its initial availability with the ESS Models E10 and E20, and then with the succeeding
F10 and F20 models, the ESS has been
the
storage server solution offering exceptional
performance, extraordinary capacity, scalability, heterogeneous server connectivity, and an
extensive suite of advanced functions to support users’ mission-critical, high-availability,
multi-platform environments. The ESS set a new standard for storage servers back in 1999
when it was first available, and since then it has evolved into the F models and the more
recent third-generation ESS Model 800, keeping up with the pace of users’ needs by adding
more sophisticated functions to the initial set, enhancing the connectivity options, and
powering its performance features.
The IBM TotalStorage Enterprise Storage Server Model 800 provides significantly improved
levels of performance, throughput, and scalability while continuing to exploit the innovative
features introduced with its preceding E and F models, such as Parallel Access Volumes,
Multiple Allegiance, I/O Priority Queuing, the remote copy functions (synchronous,
non-synchronous and asynchronous), and the FlashCopy point-in-time copy function. Also,
the heterogeneous server support characteristics (for connectivity and remote copy functions)
of previous models are continued with the ESS Model 800.
ms
High capacity scalable storage
up to 55.9 TB physical capacity
RAID 5 and
RAID 10
Arrays Across Loops
SSA loops with
faster and greater
capacity disk drives
1
12
2
3
4
5
6
7
8
9
10
11
More powerful
SMP processors
with Turbo option
Disaster Recovery solutions:
synchronous and non-synchronous
Point in time
copy solutions
zSeries
parallel I/O
Storage sharing among
different platforms:
IBM eServer servers
and non-IBM servers
IBM TotalStorage Enterprise
IBM TotalStorage Enterprise
Storage Server Model 800
Storage Server Model 800
Fault tolerant
system
TotalStorage
Powerful Seascape Architecture
Powerful Seascape Architecture

Chapter 1. Introduction
5
With the IBM TotalStorage Enterprise Storage Server Model 800, important changes have
been introduced, which dramatically improved the overall value of the ESS in the marketplace
and provide a strong base for strategic Storage Area Network (SAN) initiatives.
Third-generation hardware: ESS Model 800
The IBM TotalStorage Enterprise Storage Server Model 800 integrates a new generation of
hardware from top to bottom, allowing it to deliver unprecedented levels of performance and
throughput. Key features that characterize the performance enhancements of the ESS Model
800 are:
The ESS Model 800 is capable of delivering twice the throughput of its predecessor model
F20. Initially, with the optional Turbo feature (feature #3606), the ESS Model 800 was
capable of providing 2.5 times the throughput of the model F20, thus allowing increased
scalability and better response times. Now, the Turbo II Processor option (feature #3607),
a follow-on to the Turbo Processor (feature #3606), offers up to a 30% improvement for
high-hit ratio and high operations per second workloads as compared to the standard
processor in an ESS Model 800. Processor option feature conversions are available to
enable existing Model 800 machines to exploit this new enhancement.
64 GB cache supports much larger system configurations and increases cache hit ratios,
driving down response times.
Double the internal bandwidth provides high sequential throughput for digital media,
business intelligence, data warehousing, and life science applications.
Larger 2 GB NVS with twice the bandwidth allows greater scalability for write-intensive
applications.
Third-generation hardware provides response time improvements of up to 40% for
important database applications.
2 Gb Fibre Channel/FICON® host adapters provide doubled performance sustained and
instantaneous throughput for both open systems and zSeries environments.
64-bit ESCON® host adapters, enhanced with a faster microprocessor, providing
increased channel throughput and sequential read bandwidth.
RAID-10 can provide up to 75% greater throughput for selected database workloads
compared to equal physical capacity configured as RAID-5. While most typical workloads
will experience excellent response times with RAID-5, some cache-unfriendly applications
and some applications with high random write content can benefit from the performance
offered by RAID-10.
15,000 rpm drives provide up to 80% greater throughput per RAID rank and 40%
improved response time as compared to 10,000 rpm drives. This allows driving the
workloads to significantly higher access densities, while also experiencing improved
response times. Starting with LIC level 2.3.0., the ESS also offers rpm intermix support
among same-capacity disks.
Starting with LIC level 2.3.0, the ESS Model 800 can be configured with Arrays Across
Loops (AAL), feature number 9903. AAL is a new configuration option whereby disk arrays
are spread across two loops on the SSA device adapter pair. AAL allows you to take full
advantage of the ESS subsystem bandwidth, because a RAID array is built across two
SSA loops within a device adapter pair. Arrays Across Loops provides up to 1.8 times
improvement in single array sequential bandwidth performance.
All of this performance boost is built upon the reliable and proven ESS hardware architecture
design and unique advanced features.

6
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
1.5 IBM TotalStorage Enterprise Storage Server Model 750
The ESS Model 750 is well-suited for clients with midrange capacity and performance needs
all in an affordable package. It includes many of the functions of the ESS Model 800 and all of
its reliability. It is designed to meet the high availability requirements of mainframe and open
systems environments and is an especially good fit for the IBM ^ zSeries 8xx servers.
The ESS Model 750 consists of two clusters,
each
with a two-way processor and 4 or 8 GB
cache. It can have two to six Fiber Channel/FICON or ESCON host adapters. The minimum
storage capacity is 1.1 TB, up to a maximum storage capacity of 4 TB. A key feature is that
the ESS 750 is upgradeable, non-disruptively, to the ESS Model 800, which can grow to more
than 55 TB of physical capacity.
1.5.1 Features of the ESS Model 750
Some more details about the features of the ESS Model 750 include:
Attachment to a wide-variety of hosts, including all of the IBM^ platforms: zSeries,
pSeries, iSeries™, and xSeries®. It will also attach to a large number of different UNIX,
Linux, and Intel processor-based servers. To find the most current list of supported
servers, go to:
http://www.storage.ibm.com/disk/ess/pdf/interop.pdf
Redundant, failover hardware.
72.8 (10K rpm) and 145.6 GB (10K rpm) drives available. Each pair of eight packs must
have the same capacity and speed. Intermixing is allowed across the loops.
RAID-5 and RAID-10 are available and intermixable.
2 GB NVS and 8 or 16 GB cache.
64 non-arbitrated, pipelined paths to disks.
Supports zSeries performance enablers: PAV, Priority I/O Queueing, and Multiple
Allegiance.
Comes with LIC 2.4.0 standard at General Availability (GA).
Advanced function support for FlashCopy V1 and V2, PPRC V1 and V2.
1.5.2 Compatibility
The ESS Model 750 is compatible and can co-exist with the other models of the ESS for:
Copy Services Domain
PPRC and FlashCopy Consistency Groups
PPRC relationships
Managed under the ESS Master Console
The ESS Model 750 meets the needs of our clients for high availability, entry capacity, and
performance at a low price. It comes with a three year warranty and gives the client
investment protection, as it can be upgraded, non-disruptively, to the ESS Model 800.
Note: Arrays across loops (AAL) is not supported on the ESS Model 750.
Note: XRC (Extended Remote Copy) is not supported on the ESS Model 750.

Chapter 1. Introduction
7
1.6 ESS Model 750 and 800 hardware features
In the section, we discuss the ESS Model 750 and 800 hardware features.
Efficient cache management and powerful back end
The ESS is designed to provide the highest performance for the different type of workloads,
even when mixing dissimilar workload demands. For example, zSeries servers and open
systems put very different workload demands on the storage subsystem. A server like the
zSeries typically has an I/O profile that is very cache-friendly, and takes advantage of the
cache efficiency. On the other hand, an open system server does I/O that can be very
cache-unfriendly, because most of the hits are solved in the host server buffers. For the
zSeries type of workload, the ESS 800 has the option of a large cache (up to 64 GB) and
most importantly it has efficient cache algorithms. For the cache unfriendly workloads, the
ESS has a powerful back end, with the SSA high performance disk adapters providing high
I/O parallelism and throughput for the ever-evolving high-performance hard disk drives.
Sysplex I/O management
In the zSeries Parallel Sysplex® environments, the z/OS Workload Manager (WLM) controls
where work is run and optimizes the throughput and performance of the total system. The
ESS provides the WLM with more sophisticated ways to control the I/O across the sysplex.
These functions include parallel access to both single-system and shared volumes, and the
ability to prioritize the I/O based upon WLM goals. The combination of these features
significantly improves performance in a wide variety of workload environments.
Parallel Access Volume (PAV) and Multiple Allegiance
Parallel Access Volume and Multiple Allegiance are two distinctive performance features of
the IBM TotalStorage Enterprise Storage Server for the zSeries users, allowing them to
reduce device queue delays, which means improving throughput and response time.
I/O load balancing
For selected open system servers, the ESS in conjunction with the Subsystem Device Driver
(SDD), a pseudo device driver designed to support multi-path configurations, provides
dynamic load balancing. Dynamic load balancing helps eliminate data-flow bottlenecks by
distributing the I/O workload over multiple active paths, thus contributing to improved I/O
throughput and response time of the open system server.
2 Gb Fibre Channel/FICON host adapters
As the amount of data and transactions grow, so does the traffic over the Storage Area
Networks (SAN). As SANs migrate to 2 Gb technologies to cope with this increased amount
of data transit, so does the IBM TotalStorage Enterprise Storage Server Model 800 or Model
750 with its 2 Gb host adapters. These host adapters double the bandwidth of the previous
adapters, thus providing more throughput and performance for retrieving and storing users’
data.
64-bit ESCON host adapters
The ESCON adapters have been enhanced with a faster microprocessor that offers up to a
45% improvement in full box sequential read bandwidth and up to 10% increase in channel
throughput for random operation workloads as compared to the previous 32-bit ESCON host
adapters. When used in a PPRC configuration on both the primary and secondary ESS, up to
a 10% increase in PPRC link throughput for random write operations and sequential
bandwidth may be achieved, as compared to the previous 32-bit ESCON host adapters.

8
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Arrays Across Loops (AAL)
AAL is a new configuration option whereby disk arrays are spread across two loops on the
SSA device adapter pair. AAL allows you to take full advantage of the ESS Model 800
subsystem bandwidth, because a RAID array is built across two SSA loops within a device
adapter pair. Each SSA loop provides up to 160 MB per second non-arbitrated bandwidth for
data transfer. Providing access to two loops almost doubles the bandwidth capacity available
to a RAID rank. z/OS logical volumes or open systems LUNs are striped across the data
volumes of their associated RAID rank. Implementing AAL allows full use of this bandwidth
over twice as many paths to the disks, thus creating the possibility for multiple processes to
access a busy volume or LUN with high demand. This can help improve performance and
reduce elapsed times, especially for sequential batch and file maintenance. Overall, an AAL
configuration can provide up to a 1.8 times improvement in sequential bandwidth
performance on a single array, compared to non-AAL configurations.
AAL is only supported on the ESS Model 800, and is an optional feature to the ESS (#9903).
1.6.1 Host adapters
The following host I/O interface attachments are supported:
ESCON host adapter: The ESCON host adapter (HA) is the physical component of the
ESS used to attach the host ESCON I/O interfaces and ESCON Director ports. The
ESCON host adapter connects to an ESCON channel by means of an ESCON link and
accepts the channel command words (CCWs) from the host system. The ESS ESCON
host adapters have two ports to connect to two ESCON links.
FICON host adapter: The FICON host adapter (HA) is the physical component of the
ESS used to attach the host FICON I/O interfaces and FICON Director ports. The FICON
host adapter connects to a FICON channel by means of the FICON link and accepts the
CCWs (channel command words) from the host system. The ESS FICON host adapter is
in fact a Fibre Channel/FICON adapter card that can be configured either for FICON or for
FCP use. Both 1 Gb and 2 Gb port cards are available.
SCSI host adapter: The SCSI host adapter card is used to interface with hosts that are
using the SCSI protocol. There are different versions of SCSI, some of which can be
supported by the same adapter. The protocols that are used on the SCSI HA (the
command set) can be either SCSI-2 or SCSI-3. Each SCSI adapter has two SCSI ports.
Fibre Channel host adapter: The Fibre Channel host adapter (HA) is the physical
component of the ESS used to attach to the servers’ Fibre Channel I/O interfaces and
SAN fabric ports. The ESS Fibre Channel host adapter connects to the server Fibre
Channel I/O adapter by means of the Fibre Channel link and accepts the upper-layer
commands (more than one protocol is supported by the Fibre Channel standard) from the
host system. The ESS Fibre Channel host adapter is in fact a Fibre Channel/FICON
adapter card, and can be configured either for FICON or for FCP use. It has one 2 Gb port
for fiber connection.
1.7 Major components
The ESS can be broken down into several components. The storage server itself is
composed of two clusters that provide the facilities with advanced functions to control and
manage data transfer. Should one cluster fail, the remaining cluster can take over the
functions of the failing cluster. The overall structure is shown in Figure 1-3 on page 9.

Chapter 1. Introduction
9
Figure 1-3 ESS high level design overview
A cluster is made up of the following subcomponents:
Host adapters: Each cluster has one or more host adapters (HAs). Each host adapter
provides one or more host I/O interfaces. A host adapter can communicate with either
cluster complex.
Device adapters: Each cluster has one or more device adapters (DAs). Each device
adapter provides one or more storage device interfaces. Disk drives are attached to a pair
of device adapters, one in each cluster, so that the drives are accessible from either
cluster. At any given time, a disk drive is managed by only one device adapter.
Cluster complex: The cluster complex provides the management functions for the ESS. It
consists of cluster processors, cluster memory, cache, nonvolatile storage (NVS), and
related logic:
– Cluster processors: The cluster complex contains four cluster processors (CP)
configured as symmetrical multiprocessors (SMP). The cluster processors execute the
licensed internal code (LIC) that controls the operation of the cluster. The ESS Model
800 offers the higher speed processors and a significant performance enhancement.
– Cluster memory / cache: These are used to store instructions and data for the cluster
processors. The cache memory is used to store cached data from the disk drives. The
cache memory is accessible by the local cluster complex, by device adapters in the
local cluster, and by host adapters in either cluster.
– Non-volatile storage (NVS): This is used to store a nonvolatile copy of active written
data. Each copy of the active write data is copied to the NVS area on the other cluster
complex to obtain the redundancy. The NVS is accessible to either cluster-processor
complex and to host adapters in either cluster. Data may also be transferred between
the NVS and cache, if the algorithm detects the required data is on NVS.
N-way
N-way
SMP
SMP
Cache
Cache
Non-volatile
Non-volatile
memory
memory
NVS
Cluster
RAID
RAID
Adapters
Adapters
RAID
RAID
Adapters
Adapters
N-way
N-way
SMP
SMP
Cache
Cache
Non-volatile
Non-volatile
memory
memory
NVS
Cluster

Fault Tolerant Switch Fabric
Fault Tolerant Switch Fabric
DA
DA
Host
Host
Adapters
Adapters
Host
Host
Adapters
Adapters

10
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The disk drives provide the primary nonvolatile storage medium for any host data
stored within the ESS Storage devices. They are grouped into ranks and are managed
by the clusters.
1.7.1 Extensive capacity and scalability
The ESS model 800 offers increased performance and scalability with two new models
with 8, 16, 24, 32, or 64 GB cache options. There are various configurations possible and the
storage scalability goes up to 55.9 TB of physical capacity. There is concurrent support for all
your major server platforms, including S/390®, OS/400®, Windows® NT, Windows 2000,
NetWare, and most varieties of UNIX, including IBM AIX.
The extensive storage management capabilities include:
FlashCopy for fast data duplication
Peer-to-Peer Remote Copy (PPRC) and Peer-to-Peer Remote Copy Extended Distance
(PPRC-XD) for your synchronous and asynchronous backup and Disaster Recovery
needs
Extensive StorWatch management capability via the Web
It offers superior performance with options and innovations to meet your changing
requirements and high availability to support your e-business and other mission-critical
applications.
The ESS continues to deliver on its SAN strategy, as was previewed in the July 27, 1999,
announcement of the ESS. The ESS now provides up to sixteen 200 MB/sec. native Fibre up
to 10 km distance with long wave and 300 m with short wave. Each single port adapter
supports Fibre Channel Protocol (FCP) in a direct point-to-point configuration, point-to-point
to a switch (fabric) configuration, or Fibre Channel-Arbitrated Loop (FC-AL) in a private loop
configuration. For the complete and most up-to-date list of ESS fabric products that are
supported, refer to:
http://www.storage.ibm.com/hardsoft/products/ess/supserver.htm
The ESS Model 750 is upgradeable, non-disruptively, to the ESS Model 800.
1.8 ESS Copy Services
ESS Copy Services is an optional feature of the IBM TotalStorage Enterprise Storage Server.
It brings powerful data copying and mirroring technologies to open systems environments
previously available only for mainframe storage.
This book deals with the three main features of the ESS Copy Services for the open systems
environment (see Figure 1-4 on page 11):
Peer-to-Peer Remote Copy (PPRC)
Peer-to-Peer Remote Copy Extended Distance (PPRC-XD)
FlashCopy
ESS Copy Services Version 2 features that are added to existing ESS Copy Services
functions are also covered:
Incremental FlashCopy
Multiple Relationship FlashCopy
FlashCopy Consistency Groups
Dataset Level FlashCopy (only for S/390 systems)

Chapter 1. Introduction
11
Asynchronous Cascading PPRC and Inband FlashCopy
Also, this redbook discusses the following ESS facilities for Copy Services that became
available starting with LIC level 2.3.0:
PPRC over Fibre-Channel.
ESS Application Programming Interface enhancements for ESS Copy Services support.
Asynchronous Cascading PPRC has been enhanced to exploit failover and failback
modes in a three-site solution.
With LIC level 2.4.0 the following ESS facilities for Copy Services became available:
Asynchronous PPRC.
FlashCopy target can also be a PPRC Primary.
Figure 1-4 ESS Copy Services functions overview
1.8.1 Peer-to-Peer Remote Copy (PPRC)
PPRC is a function of a storage server that constantly updates a secondary copy of a volume
to match changes made to a primary volume. The primary and the secondary volumes can be
on the same storage server or on separate storage servers. In the case of two ESSs, the
secondary ESS can be located at another site some distance away (see Chapter 4,
“Peer-to-Peer Remote Copy (PPRC)” on page 55).
PPRC is application independent. Because the copying function occurs at the disk subsystem
level, the application has no knowledge of its existence.
TotalStorage
FlashCopy
primary site
remote site
open systems
server
PPRC
synchronous remote copy up to103 Km
local point-in-time copy
TotalStorage
TotalStorage
synchronous copy
intermediate site
asynchronous copy
incremental copy
primary
secondary
multiple copy

12
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The PPRC protocol guarantees that the secondary copy is up-to-date and consistent by
ensuring that the primary copy will be written only if the primary receives acknowledgment
that the secondary copy has been written.
1.8.2 Peer-to-Peer Remote Copy Extended Distance (PPRC-XD)
Peer-to-Peer Remote Copy Extended Distance adds flexibility to the IBM TotalStorage
Enterprise Storage Server and PPRC. PPRC-XD is a non-synchronous long-distance copy
option for both open systems and zSeries servers.
PPRC-XD can operate at very long distances, even continental distances, well beyond 300
km (maximum supported distance for Synchronous PPRC using Fibre Channel) with minimal
impact on the applications. Distance is limited only by the network and channel extenders
technology capabilities. This redbook presents an overview of the characteristics of PPRC
Extended Distance for the open system environment. For additional information and how to
use it, refer to the redbook IBM TotalStorage Enterprise Storage Server PPRC Extended
Distance, SG24-6568.
1.8.3 FlashCopy
FlashCopy makes a single point-in-time copy of a LUN. This is also known as a time-zero
copy. The target copy is available once the FlashCopy command has been processed (see
Chapter 3, “FlashCopy” on page 35). FlashCopy provides an instant or point-in-time copy of
an ESS logical volume. Point-in-time copy functions give you an instantaneous copy, or
view
,
of what the original data looked like at a specific point-in-time.
The point-in-time copy created by FlashCopy is typically used where you need a copy of
production data to be produced with minimal application downtime. It can be used for online
backup, testing of new applications, or for copying a database for data mining purposes. The
copy looks exactly like the original source volume and is an instantly available, binary copy.
1.8.4 Incremental FlashCopy (V2)
Incremental FlashCopy is a new feature of FlashCopy that is available with ESS Copy
Services Version 2. This function is to be used in conjunction with the background copy option
to track the changes on the source volume since the last FlashCopy relationship was invoked.
When this option is selected, only the tracks that have been changed on the source are
copied again to the target. The direction of the
refresh
can also be reversed, copying the
changes made to the new source (originally the target volume) to the new target volume
(originally the source volume).
1.8.5 Inband FlashCopy (V2)
The new inband management capability feature that comes with Copy Services Version 2
allows you to invoke FlashCopy on a remote site ESS. If you have two sites, one local and one
remote, which are in a PPRC relationship, a FlashCopy task on the remote site ESS can be
invoked from the primary site ESS via a PPRC inband connection.
1.8.6 Multiple Relationship FlashCopy (V2)
With Copy Services Version 2, one FlashCopy source may have up to 12 FlashCopy targets.
This gives you more flexibility, as you can initiate the multiple relationships using the same
source volume without needing to wait for other relationships to end.

Chapter 1. Introduction
13
1.8.7 FlashCopy Consistency Groups
New options are available to facilitate the creation of FlashCopy Consistency Groups. With
the FlashCopy Consistency Groups, the ESS will hold off I/O activity to a volume until the
Consistency Created task with the FlashCopy Consistency Group option is issued.
1.8.8 Data Set Level FlashCopy (V2 for OS/390® only)
With this feature, data sets in the OS/390 environment can be copied independently from the
volume level. This feature also provides the possibility for a volume to act as a source volume
and a target volume at the same time.
1.8.9 Asynchronous Cascading PPRC (V2)
Another new feature that comes with Copy Services Version 2 is Asynchronous Cascading
PPRC. The concept of the feature is that a PPRC secondary volume serves as a PPRC
primary volume for a second PPRC relationship. The intermediate site target PPRC volume
acts as the source volume for the remote site PPRC relationship. This feature gives you the
flexibility to establish asynchronous mirroring between the intermediate and remote site as
well as synchronous mirroring between the primary and intermediate site.
With LIC level 2.3.0, Asynchronous Cascading PPRC has been enhanced to exploit the
PPRC Failover and Failback modes. This can help reduce the time required to synchronize
PPRC volumes after switching between sites during planned or unplanned outages.
1.8.10 Asynchronous PPRC (V2)
Asynchronous PPRC (V2) is designed to provide a long-distance remote copy solution across
two sites using asynchronous technology. It operates over high-speed, Fibre Channel
communication links and is designed to provide a consistent and restartable copy of the data
at the remote site, created with minimal impact to applications at the local site. Compared to
Asynchronous Cascading PPRC, Asynchronous PPRC eliminates the requirement to do a
manual and periodic suspend at the local site in order to create a consistent and restartable
copy at the remote site. Asynchronous PPRC is available with LIC 2.4.0.
1.8.11 PPRC over Fibre Channel
PPRC over Fibre Channel enables the use of Fibre Channel Protocol (FCP) as a
communications link between the ESS PPRC primary and secondary machines.
1.8.12 ESS Application Programming Interface
ESS Application Programming Interface (ESS API) has been enhanced to support ESS Copy
Services configuration and use.
1.8.13 Managing ESS Copy Services
ESS Copy Services provides a Command Line Interface (CLI) as well as a Web-based
interface for setting up and managing its facilities (see Chapter 7, “ESS Command Line
Interface” on page 377). The CLI interface allows administrators to execute Java™-based
ESS Copy Services commands from a command line. The Web-based interface, a part of the
Note: To use this function, the OS/390 version should be V2R10 or higher.

14
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
TotalStorage ESS Specialist, allows storage administrators to manage Copy Services from a
browser-equipped computer (see Chapter 5, “ESS Copy Services Web User Interface prior to
LIC 2.2.0” on page 123).
The ESS Copy Services functions can be used separately as stated above or they can be
combined to produce solutions such as split mirror for remote tape vaulting backup solutions
or static point-in-time volume replication. The allowable Copy Services combinations can be
seen in Table 1-5.
Figure 1-5 Allowable combinations of copy operation
Figure 1-5 notes:
1.Valid combination for Asynchronous Cascading PPRC only (available with PPRC V2).
2.The operation is allowed. Updates of the affected extents will result in implicit removal of
the FlashCopy relationship.
3.Valid combination for Multiple Relationship FlashCopy (available with FlashCopy V2).
4.Valid combination for Data set FlashCopy (available with FlashCopy V2).
1.9 Terminology
Copy Services functions are ESS implementations of the functions described in the IBM Total
Storage Resiliency Family Core Technologies.
1.9.1 ESS Copy Services benefits
Copy Services will be of great use to customers with large IT systems, big data volumes, and
a requirement for round-the-clock data availability.
Although each has its specific features, PPRC and FlashCopy are typically used as data
backup tools for the creation of test data and for data migration. They can also be used in
Disaster Recovery scenarios (see Chapter 9, “Disaster Recovery” on page 453).
ConCopy
source
XRC
secondary
PPRC
primary
FlashCopy
target
PPRC
secondary
FlashCopy
source
XRC
primary
XRC
primary
XRC
secondary
PPRC
primary
PPRC
secondary
FlashCopy
source
FlashCopy
target
ConCopy
source
DEVICE
IS
MAY
BECOME
2
3
4
1
1
No
Yes No
No
No
No
No
No
No
No
No
No
No
No
No
Yes No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
YesYes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes Yes
No
Yes Yes
Yes
Yes
Yes
Yes YesYes
Yes
Yes
Yes
4
4
4
Yes

Chapter 1. Introduction
15
Copy Services will provide the greatest benefit to the customer who:
Needs to have disaster tolerant IT centers
Is planning to migrate data between systems
Is migrating workloads often
Has to back up large amounts of data
Needs to reduce the time the server has to be taken offline for backup
Plans to test new applications
Needs a copy of production data for data warehousing or data mining
Copy Services can be integrated with technologies such as Tivoli® Storage Manager
(formerly ADSM), Logical Volume Manager (LVM) mirroring, or SAN Data Gateway mirroring
to solve a wide variety of business issues. Other companies will most likely be selling partial
solutions as a means to solve these problems. IBM, however, with its broad portfolio of
products in this industry, has many experts available to discuss the right solution for your
business and to help you design and implement a solution that will give you the maximum
business benefit.
Advanced solutions with ESS Copy Services have been endorsed by many Independent
Software Vendors (ISVs) worldwide.

16
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
17
Chapter 2.
Implementing ESS Copy
Services
In this chapter, we introduce the terminology used when working with the ESS Copy Services
Domains.
This chapter also describes the architecture of the Copy Services Domains. It discusses
some planning issues when preparing an installation of an ESS Copy Services Domain.
2

18
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.1 Copy Services terminology
In this section, we explain the terminology used when working within an ESS Copy Services
environment.
With the LIC code Version 2.2.x, major changes have been made to the way ESS Copy
Services are implemented. So where appropriate, a LIC code version will be provided in
brackets.
2.1.1 FlashCopy Version 1 (V1) and FlashCopy Version 2 (V2)
FlashCopy is a Copy Services feature. The FlashCopy feature must be installed and activated
on the ESS before using the FlashCopy functions. The FlashCopy feature is available in two
different versions (V1 and V2). Refer to Chapter 3, “FlashCopy” on page 35 for more details
about the two versions.
An ESS with LIC level prior to 2.2.x supports only FlashCopy V1.
An ESS with LIC level 2.2.x or higher supports FlashCopy V1 and FlashCopy V2.
2.1.2 PPRC Version 1 (V1) and PPRC Version 2 (V2)
PPRC is a Copy Services feature. The PPRC feature must be installed and activated on the
ESS before using the PPRC functions. The PPRC feature is available in two different versions
(V1 and V2). Refer to Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on page 55 for more
details about the two versions.
ESS with LIC level prior to 2.2.x supports only PPRC V1.
ESS with LIC level 2.2.x or higher supports PPRC V1 and PPRC V2.
2.1.3 Copy Services Domain
All ESS subsystem clusters participating in Copy Services relationships are grouped together
in a
Copy Services Domain
. The Copy Services Domain is managed by one or two ESS
clusters designated as
active Copy Services servers
.
A Copy Services Domain is also called a Copy Services server group.
2.1.4 Active Copy Services server
The Active Copy Services server is the server that manages the Copy Services Domain. The
Active Copy Services server is running the Copy Services server software and communicates
with the Web browsers running the Copy Services WUI.
If the ESS LIC level installed is prior to 2.2.x, this would normally be the Primary Copy
Services server, unless it fails, in which case you would manually invoke the Backup Copy
Services server as the active Copy Services server.
If the ESS LIC level installed is 2.2.x and higher, you can define two active Copy Services
servers (ServerA and ServerB). Depending on your configuration, you can have the two Copy
Services servers defined active at the same time (see 2.1.10, “Dual-active server
configuration (LIC level 2.2.x or higher)” on page 20 and 2.1.11, “Mixed mode configuration”
on page 20 for more information).

Chapter 2. Implementing ESS Copy Services
19
The active Copy Services server performs the following functions:
It collects connectivity data and configuration data from all clients that are part of the Copy
Services Domain.
It saves and manages Copy Services tasks.
It sends tasks to the appropriate client.
If you have a LIC level prior to 2.2.x, when the Primary server is the active server, it copies
tasks to the Backup Copy Services server automatically.
If you have LIC level 2.2.x or higher, it copies tasks to the other Copy Services server
automatically.
It collects volumes status updates from clients.
It communicates with the Web browser.
2.1.5 Copy Services client
A Copy Services client is software that runs on each cluster in your Copy Services Domain
and performs the following functions:
It communicates configuration, status, and connectivity information to the Copy Services
server.
It executes data-copy tasks on behalf of the Copy Services server.
2.1.6 Primary Copy Services Server (LIC level prior to 2.2.x)
When using ESS LIC prior to 2.2.x, the Primary Copy Services server is a user-designated
ESS cluster in your Copy Services Domain that performs the role of the active Copy Services
server until it fails. If the active Copy Services server running on the Primary Copy Services
server fails, an operator must manually restart the Backup Copy Services server to run as the
active server.
2.1.7 Backup Copy Services Server (LIC level prior to 2.2.x)
The Backup Copy Services server is an ESS cluster in your Copy Services Domain that can
become active when the Primary Copy Services server fails. The Backup Copy Services
server has the following characteristics:
The server remains in a non-active mode, unless an operator manually restarts it in the
event of a failure.
When the server is non-active, it receives updates to the task repository from the other
server.
It runs on a different ESS cluster than the Primary Copy Services server.
It saves tasks while running as the active Copy Services server. However, saved tasks are
lost when the ESS cluster is reset to the Primary Copy Services server.
2.1.8 Copy Services server operating modes
A Copy Services server can operate in one of two modes: dual-active or single-active.
Dual-active mode is the mode of operation if the IBM TotalStorage Enterprise Storage Server
(ESS) license internal code (LIC) level is 2.2.x or higher; otherwise, single-active is the mode
of operation.

20
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.1.9 Single-active configuration (LIC level prior to 2.2.x)
The servers are running pre-LIC 2.2.x and operating in single-active mode. One server is the
Primary and the other is the Backup. During normal operation in single-active mode, only one
of the two Copy Services servers is active. The client-only ESS can be running pre-LIC 2.2.x
or LIC 2.2.x or higher. If a disaster occurs at your production site and the Primary server is at
your production site, user intervention is required. You must make the Backup server your
active server at the recovery site for all ESSs that are running pre-LIC 2.2.x.
2.1.10 Dual-active server configuration (LIC level 2.2.x or higher)
You are in a dual-active configuration when all the ESSs in a Copy Services Domain have LIC
Version 2.2.x or higher installed. Two ESS clusters can be designated to perform the role of
the active Copy Services servers. In the dual-active mode, both Copy Services servers are
active at the same time. There is no concept of a Primary or Backup Copy Services server.
ServerA is the designation for one of the Copy Services servers, and ServerB is the
designation for the other server. All the ESSs have clients registered with both servers. |n the
event of one active server failure, there is no manual action required, as the other server is
still active.
2.1.11 Mixed mode configuration
You will be in this mixed-mode when some of the ESSs in your Copy Services Domain have
pre-LIC 2.2.x code installed while the other ESSs have LIC Version 2.2.x or higher. The
following configurations are possible:
One server has LIC Version 2.2.x or higher installed and is operating in dual-active
mode.The other server has LIC version prior to 2.2.x and is operating in single-active
mode. Client-only ESSs can be running pre-LIC 2.2.x or LIC 2.2.x and higher. With this
configuration, it is better if the dual-active server is the active server (ServerA) and the
single-active server is the passive server (backup). If a disaster occurs at your production
site and the server at the production site is the dual-active server, user intervention is
required. You must make the server at the recovery site the active server for all the ESSs
that are running pre-LIC 2.2.x.
Two servers (ServerA and ServerB) are running LIC 2.2.x or higher. At least one
client-only ESS is running pre-LIC 2.2.x. If a disaster occurs at your production site and
the server at the production site is the Primary server for the ESS that is running pre-LIC
2.2.x, user intervention is required. You must make the server at the recovery site the
active server for all client-only ESSs running pre-LIC 2.2.x.
2.1.12 The ESS Command Line Interface (CLI)
The ESS provides two command line interfaces: the ESS Management CLI and the ESS
Copy Services CLI.
ESS Management CLI
The ESS Management CLI (Command Line Interface) is software that runs on the host
systems. The ESS Management CLI provides a set of commands that you can use to monitor
and to manage the ESS functions. Some commands provided by the ESS management CLI
are very useful to view the configuration of the Copy Services environment. You can include
the CLI commands into scripts to automate some of your procedures. For more details about
the useful ESS Management CLI for Copy Services, refer to Chapter 7, “ESS Command Line
Interface” on page 377.

Chapter 2. Implementing ESS Copy Services
21
ESS Copy Services CLI
The CLI (Command Line User Interface) is software that runs on the host systems. The Copy
services CLI provides a set of commands you can use to invoke the Copy Services functions
and monitor the status of the Copy Services tasks and volumes. You can include the CLI
commands into scripts to automate some of your procedures. For more details about the CLI,
refer to Chapter 7, “ESS Command Line Interface” on page 377.
2.2 Network considerations for Copy Services
There are some Network considerations when you build the Copy Services Domain.

The ESS clusters’ communications
: This communication is necessary between the active
servers and all the clients in the Copy Services Domain. This communication is done via
the Local Area Network.

Copy Services management
: The normal way to manage the Copy Services is to use the
Web User Interface. Also, you can use the CLI to invoke Copy Services functions. Both
means (Web GUI and CLI) are using network connections to the Copy Services Domain.
Note that some configuration functions cannot be performed via the CLI, for example,
creating a task is not possible via the CLI.
2.2.1 ESS cluster communication and network configuration
The active Copy Services servers and the Copy Services clients communicate together via a
TCP/IP Ethernet connection. This communication is provided by the ESSNet.
The ESSNet is the ESS access facility that IBM installs with your ESSs. The ESSNet consists
of a switch (hub) and other networking components and the dedicated IBM TotalStorage
Enterprise Storage Server Master Console (
ESS Master Console
).
In a multiple site configuration, you need a physical network connection between the sites to
connect the ESS clusters on the same LAN.
The ESS with LIC Version 2.2.x or higher installed provides the option to use inband
commands over the PPRC link to invoke FlashCopy tasks. You may use this specific option to
invoke tasks at a remote site without maintaining the LAN connection. See Chapter 3,
“FlashCopy” on page 35 for more information about how inband management works.
2.2.2 Management via the WUI and network considerations
To manage the ESS Copy Services, a Web server running in your ESS provides a Web User
Interface that you can use to manage the ESS through a Web browser. The Web browser
must be running on a workstation connected to the ESS through the IBM TotalStorage
Enterprise Storage Server Network (ESSNet).
In addition to using a Web browser on the

ESS Master Console to connect to your ESS, you
can also use a Web browser running on your own workstation, either by connecting your
workstation directly to the ESSNet hub, or by connecting your workstation to your intranet and
connecting your intranet to the ESSNet hub. If you use your own workstation, IBM
recommends that it has at least 128 MB of memory.

22
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.2.3 Management using the CLI and network considerations
In addition to the WUI, you can use the Java based Command Line User interface (CLI) to
manage your ESS and to invoke Copy Services functions. The CLI is software that is installed
and runs on a host system (see Chapter 7, “ESS Command Line Interface” on page 377).
If you want to use the CLI commands to manage the ESSs in the domain, the requirement is
that the host from which you want to invoke the commands can communicate via the Local
Area Network to all the ESS clusters in your Domain.
If you want to use the CLI commands to invoke the Copy Services functions, the only
requirement is that the host from which you want to invoke the commands can communicate
via the Local Area Network to the active Copy Services ESS cluster in your Domain.
2.3 ESS Copy Services (LIC level prior to 2.2.x)
In this section, we discuss the implementation of a Copy Services Domain when all the ESSs
have LIC version prior to 2.2.x installed.
2.3.1 Overview and requirements
Before you can use the PPRC or FlashCopy functions, you must have the appropriate feature
codes installed on the ESS (see Appendix B, “Copy Services feature codes” on page 541):
The FlashCopy license feature ordered must be equal to or greater than the total capacity
of the ESS. The FlashCopy License feature must be installed on all ESSs on which you
want to establish FlashCopy pairs. The ESS running pre-LIC 2.2.x supports only the
FlashCopy V1 feature.
The PPRC license feature must be equal to or greater than the total capacity of the ESS.
The PPRC feature must also be purchased and installed on both the primary and
secondary ESSs. The ESS running pre-LIC 2.2.x supports only the PPRC V1 feature.
You should consider the following requirements when doing your planning:
One ESS cluster in the domain must be defined as the Primary Copy Services server.
Defining another cluster as a Backup server for the Copy Services Domain is optional.
Only one FlashCopy at a time can be active on a volume, however, you can perform a
PPRC concurrently with FlashCopy on the same volume. The designated FlashCopy
target volume cannot be a primary volume in a PPRC volume pair.
The primary and the secondary volume of a PPRC pair can only be in one relationship at
a time.
The primary and secondary volumes for a PPRC pair must be an ESS type. You cannot
establish a PPRC pair from an ESS to a different external device type.
The source and target volume for a FlashCopy must reside in the same LSS of an ESS.
You need to manage PPRC using the ESS Copy Services Web User Interface, therefore,
Ethernet and TCP/IP connectivity is needed between all of the participating ESS
subsystems, and the Web browser initiating and managing the PPRC activities.
You can use the Command Line Interface to invoke the Copy Services functions from the
command line. Ethernet and TCP/IP connectivity is needed between the server hosting
the CLI software and the Copy Services servers in the Copy Services Domain.
The source and target logical volumes must be the same size, or the target must be larger
in size to establish a FlashCopy relationship.

Chapter 2. Implementing ESS Copy Services
23
The primary and secondary logical volumes must be the same size, or the secondary
must be larger in size to establish a PPRC relationship.
There are some limitations when establishing a path from one LSS to another LSS. See
Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on page 55 for path limitations.
A Copy Services Domain can handle a maximum of 2048 Copy Services pairs. This
number includes all the primary and secondary PPRC pairs plus all the source and target
FlashCopy pairs.
2.3.2 Defining the Copy Services Domain
A Copy Services Domain is a group of ESS clusters participating in Copy Services
relationships.
Up to four ESSs can reside in the same Copy Services Domain. You will choose, among the
clusters in the domain, a pair of clusters and define them as servers for the Copy Services
Domain:
One Cluster to be defined as the Primary Copy Services server (mandatory).
One Cluster to be defined as the Backup Copy Services server (optional but
recommended).
All the other clusters will be clients for the Copy Services.
Domain definitions
It is mandatory to define the Primary Copy Services server in all the ESS clusters
participating in the Copy Services Domain. This definition is performed using the ESS master
console and should be done by the IBM System Service Representative (SSR).
Optionally, a Backup server for the Copy Services can be defined on all the ESS clusters in
the Copy Services Domain. This definition is performed using the ESS master console.
The definition of the Backup Copy Services server is not mandatory, but we strongly suggest
that a Backup server be defined in your Copy Services Domain. The Primary and Backup
Copy Services servers can reside on the same ESS; however, we suggest defining the
Primary and Backup server in two different Enterprise Storage servers.
During the configuration, you must also provide a list of all the ESS Clients in the Copy
Services Domain to the Primary Copy Services server. This list is known as the
Copy
Services clients
list. You must provide the ESS serial numbers, the ESS clusters’ host names
and IP addresses of all the ESSs in your Copy Services Domain (including the Primary and
Backup server). These definitions are performed using the ESS master console and should
be done by the IBM SSR.
2.3.3 Planning for a disaster
A key decision you must make in planning for a disaster is deciding where to place your
Primary and Backup Copy Services servers in a two-site environment. There are two options:
The typical setup is that you place your Primary Copy Services server at your production
site (the place where most of your applications are running).
Another option is to place your Primary Copy Services server at your recovery site.

24
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
There are trade-offs associated with each option:
Benefits of placing your Primary Copy Services server at your production site:
If a disaster or emergency situation at your production site causes you to lose connections
to the Primary Copy Services server, you can switch control to your recovery site. The
ESS Copy Services code copies tasks synchronously between the Primary and Backup
servers. This configuration might be the desirable setup in open systems environments.
The reason for this is that the ESS Copy Services CLI uses TCP/IP links to communicate
between the host system and the Copy Services server, which may introduce network
delays. However, with this configuration, if your primary site fails, there can be delays in
bringing up applications at the recovery site.
Benefits of placing your Primary Copy Services server at your recovery site:
Choosing this configuration can gain you some efficiency in terms of recovering from a
disaster. Because the Primary server is at the recovery site, you do not have to perform
manual recovery steps to switch to a different Copy Services server. If you have planned in
advance and have created tasks for disaster recovery, then you can run those tasks and
bring your production systems back up.
See the IBM TotalStorage Enterprise Storage Server: Introduction and Planning Guide,
GC26-7294 for more information about factors you should consider in making the decision
about where to place your ESS Copy Services servers.
2.4 ESS Copy Services (LIC level 2.2.x or higher)
In this section, we discuss the implementation of the Copy Services Domain when all the
ESSs have LIC level 2.2.x or higher installed. We also describe the implementation of a Copy
Services Domain with a mixed environment (running pre-LIC 2.2.x and LIC 2.2.x or higher).
2.4.1 Overview and requirements
Before you can use the PPRC or FlashCopy functions, you must have the appropriate feature
codes installed on the ESS (see Appendix B, “Copy Services feature codes” on page 541):
The FlashCopy license feature ordered must be equal to or greater than the total capacity
of the ESS. The ESS running LIC 2.2.x or higher supports FlashCopy V1 and FlashCopy
V2 feature codes.
The PPRC license feature must be equal to or greater than the total capacity of the ESS.
The PPRC feature must also be purchased and installed on both the primary and
secondary ESSs. The ESS running LIC 2.2.x or higher supports PPRC V1 and PPRC V2
feature codes.
You should consider the following requirements when planning:
Using FlashCopy V2, you can establish multiple FlashCopy relationships at one time using
the same source volume. The maximum number of FlashCopy relationships using the
same source volume is 12. Among all the FlashCopy relationships using the same source
volume, only one can be an incremental FlashCopy (which is also introduced with
FlashCopy V2).
A secondary (target) volume of a PPRC relationship can become the primary volume of
another PPRC relationship. This possibility is provided by the Asynchronous Cascading
PPRC functionality, which comes with PPRC V2.

Chapter 2. Implementing ESS Copy Services
25
You can use one logical volume for FlashCopy and PPRC at same time. The designated
FlashCopy target volume cannot be a primary volume in a PPRC volume pair, unless you
use FlashCopy V2 and you have LIC Version 2.4.x or higher installed.
There is a possibility to use Fibre Channel links for PPRC connectivity. You must have
PPRC V2 feature installed and also the LIC level must be 2.3.x or higher.
The primary and secondary volumes for PPRC must be an ESS type. You cannot
establish a PPRC from an ESS to a different external device type.
The source and target volume for FlashCopy must reside in the same ESS. The source
and target volumes can be in different LSSs.
You need to manage PPRC using the ESS Copy Services Web User Interface, therefore,
Ethernet and TCP/IP connectivity is needed among all of the participating ESS
subsystems and the Web browser initiating and managing the PPRC activities.
You can use the Command Line User Interface to invoke the Copy Services functions from
a command line. Ethernet and TCP/IP connectivity is needed between the server hosting
the CLI software and the Copy Services servers in the Copy Services Domain.
The source and target logical volumes must be the same size, or the target must be larger
in size to establish a FlashCopy relationship.
The primary and secondary logical volumes must be the same size, or the secondary
must be larger in size to establish a PPRC relationship.
There are some limitations when establishing a path from one LSS to another LSS. See
Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on page 55 for path limitations.
When using the inband commands to invoke FlashCopy tasks, the source volume you
select for the FlashCopy at the remote site must be the secondary PPRC volume of the
PPRC pair (the PPRC secondary volume becomes the source volume in a FlashCopy
pair). FlashCopy V2 must be installed to be able to use this feature.
A Copy Services Domain can handle a maximum of 2048 Copy Services pairs. This
number includes all the primary and secondary PPRC pairs plus all the source and target
FlashCopy pairs.
2.4.2 Defining the Copy Services Domain
A Copy Services Domain is a group of ESS clusters participating in Copy Services
relationships.
Starting with LIC level 2.3.x, up to eight ESSs can reside in the same Copy Services Domain.
You will choose, among the clusters in the domain, a pair of clusters and define them as
servers for the Copy Services Domain:
One cluster to run as active ServerA (mandatory).
One cluster to run as active ServerB (optional, but recommended).
All of the other clusters will be clients for the Copy Services.
Domain definitions
It is mandatory to define the ServerA in all of the ESS clusters running LIC Version 2.2.x or
higher and participating in the Copy Services Domain. These definitions can be performed
using the WUI or using the ESS master console.
Optionally, in all of the ESS clusters running LIC 2.2.x or higher and participating in the Copy
Services Domain, you can also define a ServerB as the other active Copy Services server.
This definition is performed using the WUI or using the ESS master console.

26
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
During the configuration, you must also provide a list of all the ESS clients in the Copy
Services Domains to the ServerA and the ServerB. This list is known as the Copy Services
clients list. You must provide the ESS serial numbers, the ESS clusters host names, and IP
addresses of all the ESSs in your Copy Services Domain (including the ServerA and ServerB
clusters). The definition is performed by using the WUI or using the ESS master console.
The procedure is detailed in the Chapter 6, “ESS Copy Services Web User Interface: LIC
2.2.0 and later” on page 189.
2.5 Copy Services Domain configurations
In this section, we discuss the following operating modes:
The dual-active configuration: All the ESSs in the domain have LIC Version 2.2.x or higher
installed.
The mixed configuration: The ESSs in your domain have different versions of LIC installed
(pre-2.2.x and 2.2.x or higher). Two configurations are possible in the mixed mode.
2.5.1 Dual-active configuration
In this configuration, all the ESSs in the Copy Services Domain are running LIC 2.2.x or
higher and two ESS clusters are defined as active servers (ServerA and ServerB). Both
servers will be active at the same time. In the event of one active server failure, there is no
manual action required, as the other server is still active.
Planning for Disaster Recovery in dual-active mode
A key decision you must make in planning for a disaster is deciding where to place your
Server A and Server B in a two-site environment.
For maximum protection against a disaster, ensure that one dual-active server is the IP
address of an ESS cluster at your production site and the other dual-active server is the IP
address of an ESS cluster at your recovery site.
It is preferable that ServerB resides at the production site and ServerA at the recovery site.
Configuration example
The following is an example of a two-site configuration with three ESSs. All the ESSs are
running LIC 2.2.x or higher. Figure 2-1 on page 27 and the Table 2-1 on page 27 provide
details about this dual-active configuration. Here are some specific points about this example:
All the ESSs are running LIC 2.2.x or higher and two servers are defined as the active
servers (ServerA and ServerB). ServerA is at the remote site, and ServerB is at the local
site.
The ServerA and ServerB IP address are defined on all the clusters within the Copy
Services Domain. The clients list is defined on both ServerA and ServerB. A Domain-wide
Reset procedure was performed from ServerA to initiate the communication between the
servers and the clients.
Task definitions can be performed either by starting the Copy Services WUI with ServerA
or with ServerB.
Attention: When the two servers resume communication after a Disaster Recovery, the
task repository in ServerB is overwritten by the one in ServerA, when the ServerA
becomes active.

Chapter 2. Implementing ESS Copy Services
27
In case of a failure of ServerA or ServerB, no manual action is required, since the other
server can take over the Copy Services management.
Figure 2-1 Example of a domain
The following table shows the configuration details of the Copy Services Domain using three
ESSs with LIC level 2.2.x or higher installed.
Table 2-1 Example of configuration: dual-active servers and clients
ESS A ESS B ESS C
Site LOCAL LOCAL REMOTE
LIC level installed LIC 2.2.x or higher LIC 2.2.x or higher LIC 2.2.x or higher
Cluster 1 IP Address 10.10.1.21 10.10.1.31 10.10.1.41
Cluster 1 is...Active server
(ServerB)
Client Active server
(ServerA)
Cluster 1 Copy
Services definitions
ServerA=10.10.1.41
ServerB=10.10.1.21
and Clients list
ServerA=10.10.1.41
ServerB=10.10.1.21
ServerA=10.10.1.41
ServerB=10.10.1.21
and Clients list
and “Domain-wide
Reset” procedure
Cluster 2 IP Address 10.10.1.22 10.10.1.32 10.10.1.42
Cluster 2 is...Client Client Client
Cluster 2 Copy
Services definitions
ServerA=10.10.1.41
ServerB=10.10.1.21
ServerA=10.10.1.41
ServerB=10.10.1.21
ServerA=10.10.1.41
ServerB=10.10.1.21
ESS B
LIC level 2.2.x or higher
LOCAL site
REMOTE site
ESCON or
FCP Links
ESCON or
FCP Links
LAN
LAN
Administration
Local Console
Administration
Remote
Console
ESS A
LIC level 2.2.x or higher
Cluster 1 acting as
ServerB
ESS C
LIC level 2.2.x or higher
Cluster 1 acting as
ServerA

28
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.5.2 Mixed configuration
In this configuration, you have a mix of ESSs running pre-LIC 2.2.x and LIC 2.2.x or higher.
There are two possible configurations:
Configuration A: One server is running LIC 2.2.x or higher and operating in dual-active
mode. The other server is running pre-LIC 2.2.x and operating in single-active mode.
Client-only ESSs can be running pre-LIC 2.2.x or LIC 2.2.x and higher. With this
configuration, it is better if the dual-active server is the active server (ServerA) and the
single-active server is the passive server (backup). If a disaster occurs at your production
site and the server at the production site is the dual-active server, user intervention is
required. You must make the server at the recovery site the active server for all the ESSs
that are running pre-LIC 2.2.x.
Configuration B: Two servers (ServerA and ServerB) are running LIC 2.2.x or higher. At
least one client-only ESS is running pre-LIC 2.2.x. If a disaster occurs at your production
site and the server at the production site is the primary server for the ESS that is running
pre-LIC 2.2.x, user intervention is required. You must make the server at the recovery site
the active server for all client-only ESSs running pre-LIC 2.2.x.
Important considerations when defining the domain in configuration A
With your mixed configuration A, you cannot have, in the same domain and at the same
moment, one active Copy Services server running LIC 2.2.x or higher and one active Copy
Services server running pre-LIC 2.2.x. The single-active server must be in non-active
(passive) mode. The single-active server will be manually switched to the active mode only if
the dual-active server is not active or not available (this can happen, for example, in a
disaster recovery scenario).
Perform the following steps to put the Backup server in a passive (non-active) state:
1.Establish a browser connection to the ESS Launch panel at the single-active server.
2.Click the Tools button.
3.Perform a reset to primary procedure by clicking on the Reset to Primary button.
4.You also need to establish a browser connection to all the other ESS clusters running
pre-LIC 2.2.x (and acting as clients) and perform steps 2 and 3 to initiate the
communication with the active server.
This procedure is detailed in Chapter 5, “ESS Copy Services Web User Interface prior to LIC
2.2.0” on page 123.
Planning for Disaster Recovery in mixed configuration A
For maximum protection against a disaster, one server must be the IP address of an ESS
cluster at your production site and the other server should be the IP address of an ESS
cluster at your recovery site.
Two configurations are supported:
The dual-active server is at your production site and the single-active server is at your
recovery site.
The dual-active server is at your recovery site and the single-active server is at your
production site.
Attention: In a mixed configuration, any ESS cluster that is running pre-LIC 2.2.x must
have its primary Copy Services server defined as the IP address for ServerA.

Chapter 2. Implementing ESS Copy Services
29
Configuring mixed configuration A
On all ESSs running LIC 2.2.x or higher:
– The ESS cluster defined as ServerA is a cluster running LIC 2.2.x or higher.
– The ESS cluster defined as ServerB is a cluster running pre-LIC 2.2.x.
On the ESS running LIC prior to 2.2.x:
– The ESS cluster defined as the Primary server is a cluster running LIC 2.2.x or higher.
It must be the ServerA cluster.
– The ESS cluster defined as Backup server is a cluster running pre-LIC 2.2.x. It must be
the Backup server cluster.
In this configuration, one Copy Services server (the ServerA) operates in Dual-active mode,
while the other Copy Services server (the Backup server) operates in Single-active mode.
Be aware of the following:
The Copy Services clients with LIC version prior to 2.2.x installed still must be configured
by defining a
Primary server
and a
Backup server
.
You should define the Primary server with the IP address of the ServerA, and the Backup
server with the IP address of the Backup server cluster.
All the ESS clusters in the domain with LIC prior to 2.2.x will be only able to communicate
with one active server. You will have to manually initiate the communication to the active
server using the Reset to primary procedure. Those actions are described in Chapter 5,
“ESS Copy Services Web User Interface prior to LIC 2.2.0” on page 123.
Example of mixed configuration A
We give here an example of a two site configuration with three ESSs. Two of the ESSs are
running LIC 2.2.x or higher and one is running pre-LIC 2.2.x.
Figure 2-2 Example of a domain
ESS B
LIC level 2.2.x or higher
LOCAL site
REMOTE site
ESCON or
FCP Links
LAN
LAN
Administration
Local Console
Administration
Remote
Console
ESS A
LIC level 2.2.x or higher
Cluster 1 acting as
ServerA
ESS C
LIC level prior to 2.2.x
Cluster 1 defined as
Backup server (in
non-active mode).
ESCON or
FCP Links

30
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 2-2 on page 29 and Table 2-2 provide details about this mixed configuration A. Here
are some specific points about this example:
One cluster at the local site running LIC 2.2.x or higher is defined as ServerA. The
ServerA is in dual-active mode. One cluster running pre-LIC 2.2.x at the remote site is
defined as the Backup server. The Backup server is in single-active mode and is not
active.
The ServerA and ServerB IP address are defined on all the Clusters having LIC Version
2.2.x or higher installed. The clients list is defined on ServerA. A Reset Copy Services
procedure was performed on all clusters running LIC 2.2.x or higher or a Domain-wide
Reset procedure was performed from ServerA to initiate communication between the
ServerA and the LIC 2.2.x or higher clients.
In each of the ESS clusters running pre-LIC 2.2.x at the remote site, the primary server is
defined with the IP address of the ServerA. The Backup server is defined with the IP
address of one cluster running pre-LIC 2.2.x. A Reset to Primary procedure was
performed to initiate the communication with the active server and to put the Backup
server in non-active mode.
Task definitions must be performed by starting the Copy Services WUI with ServerA
because the server defined as ServerB is not active. The tasks that are not supported by
the Backup server will exist only on the ServerA and their task name will be appended with
~## characters (where ## is a sequence starting from 00 to 99).
In case of a failure of ServerA, we have to perform a Reset to Backup procedure on all
ESS clusters running pre-LIC 2.2.x. After the procedure, the single-active server (the
Backup server) will be the active server for Copy Services at the remote site. You can run
existing tasks and create and run new tasks from the active Backup server. The new tasks
will be lost after the recovery of the ServerA
Table 2-2 Example of mixed configuration
ESS A ESS B ESS C
Site LOCAL LOCAL REMOTE
LIC level installed 2.2.x or higher 2.2.x or higher Prior to 2.2.x
Cluster 1 IP Address 10.10.1.101 10.10.1.111 10.10.1.121
Cluster 1 is...Active server
(Server A)
Client Non-active server
(Backup server)
Cluster 1 Copy
Services definitions
ServerA=10.10.1.101
ServerB=10.10.1.121
and Clients list
and Reset Copy
Services procedure
ServerA=10.10.1.101
ServerB=10.10.1.121
and Reset Copy
Services procedure
Primary=10.10.1.101
Backup=10.10.1.121
and Reset to Primary
procedure
Cluster 2 IP Address 10.10.1.102 10.10.1.112 10.10.1.122
Cluster 2 is...Client Client Client
Cluster 2 Copy
Services definitions
ServerA=10.10.1.101
ServerB=10.10.1.121
and Reset Copy
Services or
Domain-wide Reset
procedure
ServerA=10.10.1.101
ServerB=10.10.1.121
and Reset Copy
Services procedure
Primary=10.10.1.101
Backup=10.10.1.121
and Reset to Primary
procedure

Chapter 2. Implementing ESS Copy Services
31
Configuring mixed configuration B
On all ESSs running LIC 2.2.x or higher:
– The ESS cluster defined as ServerA is a cluster running LIC 2.2.x or higher.
– The ESS cluster defined as ServerB is a cluster running LIC 2.2.x or higher.
On the ESS running LIC prior to 2.2.x:
– The ESS cluster defined as the Primary server is a cluster running LIC 2.2.x or higher.
It must be the ServerA cluster.
– The ESS cluster defined as the Backup server is running LIC 2.2.x or higher. It must be
the ServerB cluster.
Be aware of the following:
The Copy Services clients with pre-LIC 2.2.x still must be configured by defining a
Primary server
and a
Backup server
. You should define the Primary server with the IP
address of the ServerA, and the Backup server with the IP address of the ServerB cluster.
All the ESS clusters in the domain with LIC prior to 2.2.x will only be able to communicate
with one active server. You will have to manually initiate the communication to the Primary
server using the Reset to primary procedure. Those actions are described in 5.3.3,
“Restarting ESS Copy Services” on page 128.
The active ServerB that is defined as the Backup server for pre-LIC 2.2.x clients will not be
able to get the pre-LIC 2.2.x client configuration, unless you perform a Reset to Backup
procedure on pre-LIC 2.2.x clients. But then, there is no communication anymore from the
pre-LIC 2.2.x clients to the ServerA (defined as the Primary server for pre-LIC 2.2.x
clients).
Example of mixed configuration B
We give here an example of a two-site configuration with 3 ESSs. Two of the ESSs are
running LIC Version 2.2.x or higher and one ESS is running LIC prior to 2.2.x.
Figure 2-3 Example of a domain
ESS B
LIC level prior to 2.2.x
LOCAL site
REMOTE site
ESCON or
FCP Links
LAN
LAN
Administration
Local Console
Administration
Remote
Console
ESS A
LIC level 2.2.x or higher
Cluster 1 acting as
ServerA
ESS C
LIC level 2.2.x or higher
Cluster 1 acting as
ServerB
ESCON or
FCP Links

32
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 2-3 on page 31 and Table 2-3 provide details about this mixed configuration B. Here
are some specific points about this example:
ServerA and ServerB are active on two clusters running LIC Version 2.2.x or higher.
ServerA is at the local site, and ServerB is at the remote site.
The ServerA and ServerB IP addresses are defined on all the clusters with LIC level 2.2.x
or higher installed. The clients list is defined on both ServerA and ServerB. A
Domain-wide Reset procedure was performed from the ServerA to initiate the
communication between the servers and the LIC 2.2.x or higher clients.
In the ESS running pre-LIC 2.2.x, the two clusters are running the client software. The
cluster acting as ServerA is defined as the Primary server in each cluster. The ServerB is
defined as the Backup server in each cluster. A Reset to Primary procedure was
performed in each cluster to initiate the communication between the pre-LIC 2.2.x clients
and ServerA.
We decided to configure the ServerA at the local site because the ESS B with pre-LIC
2.2.x installed is also at the local site. This will avoid any manual action if the
communication is lost with the remote site.
Task definitions must be performed by starting the Copy Services WUI with ServerA
because ServerB cannot get the configuration of the ESS running the pre-LIC 2.2.x client
software.
In case of a failure of ServerA, we have to perform a Reset to Backup procedure on the
ESS B clusters running pre-LIC 2.2.x. After the procedure, we will start the Copy Services
WUI with ServerB and we will be able to run existing tasks, and create tasks and run new
tasks from ServerB. The new tasks will be lost after the recovery of the ServerA.
Table 2-3 Example of configuration: dual-active servers and pre-LIC 2.2.x clients
2.5.3 Compatibility matrix
When you define the Copy Services Domain, you must provide only a pair of servers.
Table 2-4 on page 33 describes the supported Copy server pairs configuration in a Copy
Services Domain.
ESS A ESS B ESS C
Site LOCAL LOCAL REMOTE
LIC level installed 2.2.x or higher Prior to 2.2.x 2.2.x or higher
Cluster 1 IP Address 10.10.1.21 10.10.1.31 10.10.1.41
Cluster 1 is...Active server
(Server A)
Client Active server
(ServerB)
Cluster 1Copy
Services definitions
ServerA=10.10.1.21
ServerB=10.10.1.41
and Clients list
and Domain Wide
reset procedure
Primary=10.10.1.21
Backup=10.10.1.41
and Reset to Primary
procedure
ServerA=10.10.1.21
ServerB=10.10.1.41
and Clients list
Cluster 2 IP Address 10.10.1.22 10.10.1.32 10.10.1.42
Cluster 2 is...Client Client Client
Cluster 2 Copy
Services definitions
ServerA=10.10.1.21
ServerB=10.10.1.41
Primary=10.10.1.21
Backup=10.10.1.41
and Reset to Primary
procedure
ServerA=10.10.1.21
ServerB=10.10.1.41

Chapter 2. Implementing ESS Copy Services
33
To read this table, select, in the first row and in the first column, the two kinds of Copy
Services’ servers you want to define in your domain. Then, using the intersection between the
column and the row, you can check if this server pair is supported.
Table 2-4 Supported pairs of server in the Copy Services Domain
2.5.4 Task management
The Copy Services servers running LIC Version 2.2.x or higher will act in the following ways
regarding the Task repository:
At Initialization (when the servers register with each other):
– The task repository in ServerA will overwrite the task repository in ServerB for the
tasks that are supported by both servers.
– ServerA and ServerB will maintain the tasks that are not supported by the other server.
In dual-active mode, tasks that can only be supported by ServerA, but not Server B,
exist only on ServerA. Similarly, tasks that can only be supported by ServerB, but not
ServerA, only exist on ServerB. These tasks are saved in the repository by appending
the characters ~## to the end of the task name, where ## is a sequence number from
00 to 99.
If you modify a task when two servers are running and communicating with each other
(normal configuration and operation):
– Tasks that are supported by both servers will be stored and maintained in both servers.
– Tasks that are supported by only the initiating server will be saved with the names
appended ~## in the initiating server only.
– If one or more tasks in a group task are not supported by the other server or their name
has ~xx appended, all tasks in that group that are supported by both servers will be
cloned. All the tasks within that group will have names with ~## appended. The group
task name also has ~## appended.
If you perform a LIC Upgrade on one Server:
– Tasks with ~## appended are not automatically updated in the upgraded server.
– Task with ~## appended must be manually renamed to remove the ~## to cause it to
be replicated to the other server.
ServerA ServerB Primary Backup None
ServerA NO YES NO YES YES
ServerB YES NO YES
a
a. This solution is not recommended but is allowed for a short period of time (migration, for
example). Please note that the Primary server task repository will be sent to ServerB correctly;
however, ServerB tasks will not be sent to the Primary server.
NO
NO
Example of allowed server pairs in a Copy Services Domain:
You cannot have two different ServerAs in the same Domain.
It is allowed to have one ServerA and one Backup server.
It is not allowed to have one ServerA and one Primary server.
It is allowed to have only one ServerB defined in the domain (the None column refers to
“no other cluster defined as a Copy Services Server”).

34
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.5.5 Inband management considerations
The ESS with LIC level 2.2.x or higher installed allows you to invoke some FlashCopy
commands via inband commands. That means that the active Copy Services server invokes
the FlashCopy task to a client using a PPRC link. When using inband commands, the network
connection to the client ESS is not used. This option will be useful, for example:
If you lose the network connection between the sites.
If the distance between the two sites makes it difficult or impossible to maintain a network
connection between the sites.
It is used by Asynchronous PPRC, which comes with PPRC V2 and LIC level 2.4.x or
higher.
You should consider the following points when planning to use inband commands for
FlashCopy management:
The source volume you select for the FlashCopy at the remote site must be the secondary
PPRC volume of a PPRC pair. (The PPRC secondary volume becomes the source
volume in a FlashCopy pair.)
The network connection is mandatory when you define the tasks.
There is no support for Inband queries. You cannot get the FlashCopy volume’s
information and status. For example, you would have no clue if the background copy for
an Inband FlashCopy completed on a pair unless the network connection is set up
between the Copy Services server and the ESS hosting the FlashCopy pair.
2.5.6 Migration from pre-LIC 2.2.x to LIC 2.2.x or higher
When migrating one or more ESSs in a domain, ServerA should be defined as the old
Primary and ServerB as the old Backup. The pre-LIC 2.2.x ESSs should remain unchanged.
This will insure that the task definitions are kept during the migration.
You should terminate all FlashCopy relationships before upgrading an ESS with this new LIC.
The FlashCopy relationships are not maintained during the upgrade from pre-LIC 2.2.x to LIC
2.2.x or higher. You do not have to terminate the PPRC relationships, since active PPRC
relationships are maintained. You should not define new PPRC relationships until the
migration process is completed.
Here are the different points you have to consider when planning for a migration of the ESS
LIC levels in a Domain from pre-2.2.x to 2.2.x or higher:
Check in 2.5.3, “Compatibility matrix” on page 32 for the supported Copy services server
pair configuration.
Check in 2.5.4, “Task management” on page 33 for the behavior of each server regarding
the task management. Remember that when a ServerA becomes active, it overwrites the
task repository on a ServerB.
You can define the
old
Backup server as the new ServerA.
If you want to define a ServerA that is not an
old
Primary or Backup server, you can
temporarily change the server definition for the Backup server in the domain. The
temporary definition would be that the temporary Backup server will be the ServerA after
the migration. Restart the WEB Copy Services after the temporary definition. In this way,
the tasks in the Primary will be transferred to the new ServerA. After that, one can define
the final configuration. This actually can be used as the migration steps to keep the tasks
for mixed pre-LIC 2.2.x and LIC 2.2.x or higher ESSs.

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
35
Chapter 3.
FlashCopy
Today, more than ever, organizations require their applications to be available 24 hours per
day, seven days per week. They require high availability, minimal application downtime for
maintenance, and the ability to perform data backups with the shortest possible application
outage.
The prime reason for data backup is to provide protection in case of source data loss due to
disaster, hardware failure, software failure, or user errors.
Data copies can also be taken for the purposes of program testing or data mining by
database query applications. However, normal copy operations take a long time, requiring the
prime application to be offline. With the need for 24x7 data processing, there is a need to
have an instant copy of the data.
FlashCopy allows you to move effectively towards such solutions.
In this chapter, we discuss the ESS FlashCopy function, also known as IBM TotalStorage
FlashCopy.
3

36
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
3.1 Overview
FlashCopy provides an instant or point-in-time copy of an ESS logical volume. Point-in-time
copy functions give you an instantaneous copy, or
view
, of what the original data (source)
looked like at a specific point in time. This is the so-called
time-zero copy
.
When a FlashCopy task is invoked, the process of establishing the FlashCopy pair and
creating the necessary control bitmaps takes only a few seconds to complete. Thereafter, you
have access to a time-zero copy of the source volume. As soon as the pair has been
established, you can read and write to both the source and the target volumes.
The point-in-time copy created by FlashCopy is typically used where you need a copy of the
production data to be produced with minimal application downtime. It can be used for online
backup, testing of new applications, or for creating a database for data-mining purposes. The
copy looks exactly like the original source volume and is an instantly available, binary copy.
See Figure 3-1 for an illustration of FlashCopy concepts.
Figure 3-1 FlashCopy concepts
In FlashCopy V1, the target volume is restricted to be in the same logical subsystem (LSS) as
the source volume. The source and target volumes can be on the same or on different arrays
(also on the same or different SSA loops within one DA pair), but only if they are part of the
same LSS. In FlashCopy V2, which we discuss later in this chapter, the LSS constraint has
been removed, so it is possible to have the source and the target volumes in different logical
subsystems (LSS) within a single ESS.
3.2 FlashCopy essentials
When you set up the FlashCopy, a relationship is established between the source and the
target volume and a bitmap of the source volume is created. Once this relationship is
established and the bitmap created, the target volume can be accessed as though all the data
had been physically copied. While a relationship between the source and target volume
exists, a background process copies the tracks from the source to the target.
FlashCopy provides a Time Zero copy
FlashCopy command issued
Time
Copy immediately available
Read and write to both source
and copy possible
Write
Read
When copy is complete,
relationship between
source and target ends
Source Target
Read
Write
T
0

Chapter 3. FlashCopy
37
The relationship ends when the physical background copy process has completed or when
the relationship is ended by the issuing of the Withdraw FlashCopy pair task.
At the time when the FlashCopy is started, the target volume is, in a sense, empty. The
background copy process copies data from the source to the target. The FlashCopy bitmap
keeps track of which data has been copied from the source to target. If an application wants
to read some data from the target that has not yet been copied to the target, the data is read
from the source; otherwise, the read is satisfied from the target volume. When the bitmap is
updated for a particular piece of data, it signifies that the source data has been copied to the
target volume. Further updates to the same area are ignored by FlashCopy. This is the
essence of the time-zero point-in-time copy mechanism.
Before data is destaged to a track on the source that has not yet been copied, the original
track is copied to the target volume. Reads that are subsequently directed to this track on the
target volume are now satisfied from the target volume instead of the source volume. After
some time, all tracks will have been copied to the target volume, and the FlashCopy
relationship will end, unless the Persistent FlashCopy option was used.
You cannot create a FlashCopy on one type of operating system and make it available to a
different operating system, unless they support the same structure of storing data. You can
make the target available to another host running the same type of operating system. Also,
remember that the FlashCopy creates an exact binary copy of the source volume. So if you
want to access the target from the same system as the source, special conditions must be
met for different platforms (see Chapter 8, “Open systems specifics” on page 407 for more
details).
3.3 FlashCopy V2 enhancements
FlashCopy V2 includes support for all previous FlashCopy functions. There are also new
enhancements available for the open-systems environment. They include:
Elimination of the logical subsystem constraint
Multiple Relationship FlashCopy
Incremental FlashCopy with Reverse Restore option
FlashCopy Consistency Groups
Inband commands over the PPRC link
Establish time improvement
3.3.1 FlashCopy across different logical subsystems (LSS)
This is a new feature in FlashCopy V2. From now on, the source and the target volumes can
span logical subsystems. We can FlashCopy to any volume within a single ESS.
This gives us more flexibility and simplified capacity management and administration. For
more information about logical subsystems, see Appendix C, “Logical subsystems” on
page 549.
3.3.2 Multiple Relationship FlashCopy
This feature provides the capability for the source volume to have FlashCopy relationships
with multiple target volumes simultaneously. This allows you to initiate up to 12 FlashCopy
relationships on a given source volume, without needing to first wait for or cause the previous
relationships to end.

38
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The limitations using Multiple Relationship FlashCopy are:
The target volume can only have one source volume.
The target volume cannot be used as the source volume at the same time (for another
FlashCopy relationship).
The source and the target volumes can be spread across any LSS within a single ESS.
3.3.3 Incremental FlashCopy and the Reverse Restore option
This feature provides the capability to
refresh
a volume involved in a FlashCopy relationship.
With Incremental FlashCopy, the initial relationship between the source and the target
volume is maintained and read/write operations are allowed on both the source and the target
(see 6.11.6, “Incremental FlashCopy” on page 240). When a subsequent FlashCopy establish
is initiated, only the data that has changed since the last time-zero or incremental copy is
copied. This helps reduce background copy completion time when only a subset of the data
has changed.
The direction of the
refresh
can also be reversed, in which case the volume previously defined
as the target becomes the source for the volume previously defined as the source (and is now
the target). Again, only the data that has changed is copied to the volume previously defined
as the source. See Figure 3-2 for an illustration of Incremental FlashCopy concepts.
Figure 3-2 Incremental FlashCopy concepts
In Figure 3-2, we have two volumes: a source volume and a target volume. After establishing
the FlashCopy relationship between these two volumes with the Start Change Recording
option enabled (the Persistent FlashCopy option is automatically selected), the ESS creates
the control bitmap for each volume in this relationship. All tracks that change on either of the
volumes are marked in the corresponding bitmap. The Incremental FlashCopy is used to
create a point-in-time copy of the source volume, but not by copying the entire volume again.
It updates the target volume only with data that has changed on the source volume since the
last point-in-time copy. After the Establish FlashCopy pair task with Increment FlashCopy
option is run, the tracks that have changed on the target are overwritten by the corresponding
Source Target
Write
Read
Write
Read
Incremental FlashCopy
Initial FlashCopy
relationship established
with change recording and
persistent copy options
Control bitmap for each
volume created
Tracks changed on the
target are overwritten by
the corresponding tracks
from the source
Possible reverse operation,
the target updates the
source
Incremental FlashCopy
started
Tracks changed on the
source are copied to the
target

Chapter 3. FlashCopy
39
tracks from the source. Thereafter, the tracks that have changed on the source are copied to
the target (tracks that have already been copied will not be copied again). This ensures that
the target volume becomes an updated point-in-time copy of the source volume without the
need to copy the entire volume again. Figure 3-3 shows an example of how Incremental
FlashCopy works.
Figure 3-3 Incremental FlashCopy example
In Figure 3-3, we have two volumes: A is the source volume, and B is the target volume. We
establish the initial FlashCopy pair with the Start Change Recording option enabled. Two
bitmaps are created, one to record which tracks have been updated on volume A, the second
to record tracks updated on volume B. Each of the volumes have nine tracks: A1 to A9 on
volume A, B1 to B9 on volume B. After establishment of the FlashCopy with the Start Change
Recording option enabled, tracks A1, A3, A4, and A8 have changed on volume A (marked as
A1c, A3c, A4c, and A8c in Figure 3-3) and tracks B1, B2, and B5 have changed on volume B
(marked as B1c, B2c, and B5c in Figure 3-3). After you run the Establish FlashCopy pair
task with the Increment FlashCopy option from A to B, the target will contain the following
tracks: A1c, A2, A3c, A4c, A5, B6, B7, A8c, and B9. Tracks that have not changed (marked
with the “equal to” sign in Figure 3-3) are not copied.
3.3.4 FlashCopy Consistency Groups
New options are available to facilitate the creation of FlashCopy Consistency Groups. With
the FlashCopy Consistency Groups, the ESS will hold off I/O activity to a volume until the
Consistency Created task with the FlashCopy Consistency Group option is issued (see
Figure 3-4 on page 40).
The Freeze FlashCopy Consistency Group option for the Establish FlashCopy pair task
(see Chapter 6, “ESS Copy Services Web User Interface: LIC 2.2.0 and later” on page 189)
provides the capability to ensure data consistency across multiple FlashCopy volumes in an
LSS, across LSSs, and even across multiple ESSs (remember that FlashCopy cannot be
used to copy data between two or more ESSs, so each source volume must have its target
Note: The background copy process must be finished before you use the Reverse
Restore option.
A1
A2
A3
A4
A5
A6
A7
A8
A9
B1
B2
B3
B4
B5
B6
B7
B8
B9
A1c
A2
A3c
A4c
A5
A6
A7
A8c
A9
B1c
B2c
B3
B4
B5c
B6
B7
B8
B9
A1c
A2
A3c
A4c
A5
A6
A7
A8c
A9
A1c
A2
A3c
A4c
A5
B6
B7
A8c
B9
Volume A
Volume B
Volume A
Volume B
Volume A
Volume B
Read/Write operations to A and B
Initial FlashCopy established:
A1=B1.....A9=B9
Volumes A and B changed:
A1c not equal to B1c; A2 not
equal to B2c; A3c not equal to
B3; A6=B6
Volume B exact copy of Volume A,
only changed tracks copied
Incremental FlashCopy started
Incremental FlashCopy example
= = =

40
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
within the same ESS). It causes volumes to remain in a
Queue Full

(QF)
condition, until the
Consistency Created task with the FlashCopy Consistency Group option is run.
The QF status signals to the host that the tagged SCSI command queue is full and that the
I/O request has not been placed in the queue. Then the host system will attempt to requeue
the requests in the logical unit’s device queue.
In other words, FlashCopy Consistency Groups provide the capability to temporarily queue (at
the host’s level) subsequent write operations to the FlashCopy source volumes that are part
of the Consistency Group, until the QF condition is reset by the Consistency Created task or
the time-out value expires (the default is two minutes; see “Consistency Group timeout” on
page 212). To learn more about how to reset the QF condition, see 6.11.10, “Thawing a
FlashCopy Consistency Group” on page 255.
Figure 3-4 Consistency Groups with FlashCopy
The default time-out value can be changed using the ESS Copy Services Web User Interface
(WUI) (see 6.6.4, “LSS properties” on page 212). Consistency groups can be used to help
create a consistent point-in-time copy across multiple volumes, and even across multiple
ESSs.
3.3.5 Inband commands over the PPRC link
In a PPRC environment, commands to manage FlashCopy at the remote site can now be
issued from the local site and transmitted over PPRC links (see 6.11.11, “Inband FlashCopy”
on page 256).
FlashCopy commands are issued to the primary volume of a PPRC pair at the local site. The
PPRC pair acts as a conduit to the remote site for the execution of the command at the
remote site. The source volume that you select for the FlashCopy operation at the remote site
must be
a secondary PPRC volume of the PPRC pair.
Note: If the Consistency Group source volumes are used with a journaled file system (like
AIX JFS) and the source LUNs are not unmounted before running FlashCopy, it is likely
that fsck will have to be run on the target volume. This is because the file system
meta-data is written directly to the source volume without using any system buffers, unlike
the data, which is usually written to the source using a periodic
sync
operation, which
flushes the buffers to the source volume.
FlashCopy
zSeries
CG
zSeries
CG
Ensures data consistency accross multiple volumes,
multiple LSS, and even across multiple ESSs
Freeze comand holds off I/O activity to the volumes,
which are part of the Consistency Group
Volumes in "long-busy" condition until reset or timeout
Reset done by "Run Consistency Group after a Freeze
command" task (one per LSS)
Consistency Groups with FlashCopy

Chapter 3. FlashCopy
41
This new function eliminates the need for a network connection to the remote site solely for
the management of FlashCopy. Remember that you still need the network connection to the
remote site for the purpose of creating the Inband FlashCopy task, because you must be able
to select the volumes that you intend to use for the FlashCopy relationship.
3.3.6 Establish time improvement
Performance improvements in FlashCopy V2 are designed to provide up to a 10 times
reduction in the time required to complete the Establish FlashCopy pair task. With this
significant reduction in establish time, operational interruption is further minimized, and the
benefits of FlashCopy can be extended into new application environments.
3.4 FlashCopy tasks and options
In this section, we discuss various options for both FlashCopy V1 and FlashCopy V2. There
are two basic types of FlashCopy tasks that can be used with various options to create the
ESS Copy Services tasks:
Establish FlashCopy pair: This type of task establishes a FlashCopy pair between
source and target volume, so they are in the FlashCopy relationship. To learn more, see
6.11.1, “Establishing a FlashCopy pair” on page 226.
Withdraw FlashCopy pair: This type of task is used to withdraw the FlashCopy pair, so
the relationship between source and target volume ends. To learn more, see 6.11.3,
“Withdrawing a FlashCopy pair” on page 232.
3.4.1 FlashCopy V1 establishment options
With FlashCopy V1, you can use the following options for the Establish FlashCopy pair task
type to create the ESS Copy Services tasks:
No background copy
You can suppress the background copy process using the No background copy (NOCOPY)
option. This may be useful if you need the copy only for a short time, such as making a
backup to tape. If you perform FlashCopy with the NOCOPY option, data will only be copied
from the source volume to the target volume if a track on the source volume is modified.
Generally the FlashCopy relationship between source and target volumes remains indefinitely
and has to be broken manually. The best way to break the relationship is to use the Withdraw
FlashCopy pair task.
If you need to create a permanent physical copy of the data for backup or disaster recovery
purposes, you can perform a Withdraw FlashCopy pair operation with the FlashCopy Start
Note: You must first establish PPRC paths and the PPRC pair from the local ESS logical
subsystem to the remote ESS logical subsystem containing the volumes that are used for
the Inband FlashCopy task.
Note: In this chapter, we will use references to other chapters, mainly to the ESS Copy
Services Web User Interface (WUI), which is now divided into two separate chapters, as
there are two versions of the ESS Copy Services and WUI. The reader should refer to the
sections according to the section names specified, but in the chapter corresponding to the
version of the WUI used.

42
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Background Copy option (see “FlashCopy Start Background Copy” on page 43). All data will
be copied from the source volume to the target volume. After the copy is complete, the
FlashCopy relationship is terminated, unless the FlashCopy relationship is persistent, in
which case, the relationship remains. Refer to “Persistent FlashCopy” on page 42 for further
information.
Accelerated destage mode
Select this option to cause a FlashCopy source volume track, which is being modified and is
in a FlashCopy relationship, to be destaged from cache sooner than it would be if normal
cache algorithms were applied. This feature minimizes (for the source volume) the number of
modified tracks that are resident in cache. With FlashCopy Version 1, this option might (under
specific circumstances) speed up the completion of the background copy process.
This feature has no effect with FlashCopy Version 2, as the cache algorithms have changed.
Permit establish if target is online
Select this option if you want to establish a FlashCopy relationship, even if the target is online,
to the S/390 or zSeries host.
Persistent FlashCopy
If you want to retain the FlashCopy relationship after the background copy completes, you can
use the Persistent FlashCopy option. The FlashCopy relationship between the source and
the target volume remains indefinitely and must be broken by a Withdraw FlashCopy pair
task (see 5.10.3, “Withdrawing a FlashCopy pair” on page 155).
This has no impact on host performance once the background copy is finished. The
Persistent FlashCopy option is usually used to protect the target volume from being
overwritten by another ESS Copy Services task. With FlashCopy Version 2, it is a prerequisite
for using Incremental FlashCopy.
3.4.2 FlashCopy V1 withdraw options
With FlashCopy V1, you can use the following options for the Withdraw FlashCopy pair task
type to create the ESS Copy Services tasks.
FlashCopy withdraw to target
Use this option if you want to remove the I-am-a-copy icon from the target volume (5.10.3,
“Withdrawing a FlashCopy pair” on page 155). Typically, you remove the indicator because
you decide it is no longer necessary to mark the volume as a copy of another volume. This
function might be useful also in other situations (like if there is a source volume missing). See
“Withdraw FlashCopy to the target” on page 157 for further details.
Note: After a FlashCopy pair is established with the No background copy option, an
automatic withdrawal of the FlashCopy pair is caused when all sectors or cylinders on the
source volume are modified. This rarely occurs, as it is unlikely that the whole contents of a
disk volume is modified.
Note: This option does not apply to the open-system volumes.
Note: To use this option, you must select the target volume of the established FlashCopy
pair as a source and also as a target in the Volumes panel of the ESS Copy Services Web
User Interface (WUI).

Chapter 3. FlashCopy
43
FlashCopy Start Background Copy
This option allows a customer to change a FlashCopy from a NOCOPY to a COPY, which will
speed up the completion of creating the target volume. Only the remaining tracks are copied
to the target volume.
This is an option for the Withdraw FlashCopy pair task type. See 5.10.4, “FlashCopy Start
Background Copy” on page 159 for further details.
3.4.3 FlashCopy V2 establishment options
In addition to FlashCopy V1 options, with FlashCopy V2, you can use the following options for
the Establish FlashCopy pair task type to create the ESS Copy Services tasks.
Inband command
Select this option to establish a FlashCopy relationship between two volumes at a remote
ESS using inband commands; the FlashCopy source volume must be the PPRC secondary
volume (see 6.11.11, “Inband FlashCopy” on page 256). Inband commands are issued to a
primary volume of a PPRC pair at the local ESS and sent across PPRC paths (acting as a
conduit) to a remote ESS to establish FlashCopy at the remote site.
This eliminates the need for the network connection to the remote site solely for FlashCopy
management. Remember that you still need the network connection to the remote site for the
purpose of creating the Inband FlashCopy task, because you must be able to select the
volumes which you intend to use for the FlashCopy relationship.
Freeze FlashCopy Consistency Group
This option causes the source volume(s) to go into the Queue Full condition. All writes to the
volume(s) are queued by the host and are written after the condition is reset, either using the
Consistency Created task with the FlashCopy Consistency Group option or by the
time-out value, which has a default value of two minutes. See 6.6.4, “LSS properties” on
page 212 to see how to modify this value. Consistency groups can be used to help create a
consistent point-in-time copy across multiple volumes, and even across multiple ESSs.
Inhibit writes to target
If you select this option, any write operation to the target volume of the FlashCopy pair will fail
as long as the FlashCopy relationship exists. This is used if you want to protect the target
volume from any updates from the host side.
Be careful while accessing the target volume from the same host system as the source while
write operations to the target volume are inhibited. This might cause serious problems
(depending on the operating system or volume manager software used), as information which
has to be changed on the target volume to differentiate between the two volumes (because
the target volume is an exact binary copy of the source volume) is not written.
Note: To use this option, you must select the source volume of the established FlashCopy
pair as a source and also as a target in the Volumes panel of the ESS Copy Services Web
User Interface (WUI).
Note: You must first establish PPRC paths from the local ESS logical subsystem to the
remote ESS logical subsystem containing the volumes that are used for the Inband
FlashCopy task.

44
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Start Change Recording
This option is used to create the bitmaps that record the changed tracks on both volumes
within a FlashCopy pair. It is a prerequisite when you want to use the Incremental FlashCopy.
This option has to be selected (Persistent FlashCopy option is autoselected) while you are
establishing the initial FlashCopy pair that you intend to use with the Incremental FlashCopy
(see 6.11.6, “Incremental FlashCopy” on page 240).
If you want to run subsequent Incremental FlashCopy tasks on the same volume pair, this
function has to be selected each time you increment the FlashCopy.
Incremental FlashCopy
This option is used for incrementing the existing FlashCopy pair. The option ensures that only
the changed tracks are copied from the source to the target volume (for more details see
3.3.3, “Incremental FlashCopy and the Reverse Restore option” on page 38). Before using
this option, there must be an established FlashCopy pair with the Start Change Recording
option available.
Reverse Restore
Use this option to reverse an established FlashCopy pair. After establishing the FlashCopy
with the Reverse Restore option enabled, the volume previously defined as a source
becomes a target for the volume previously defined as a target (and now is the source). To be
able to use this option, there must be an established FlashCopy pair with the Start Change
Recording option available. For more detailed information about how this function works,
refer to 3.3.3, “Incremental FlashCopy and the Reverse Restore option” on page 38.
3.4.4 FlashCopy V2 withdraw options
In addition to FlashCopy V1 options, with FlashCopy V2, you can use the following options for
the Withdraw FlashCopy pair task type to create the ESS Copy Services tasks.
Inband command
Select this option to withdraw the FlashCopy relationship between two volumes at a remote
ESS using inband commands (see 6.11.11, “Inband FlashCopy” on page 256). Inband
commands are issued to a primary volume of a PPRC pair at the local ESS and sent across
PPRC paths (acting as a conduit) to a remote ESS to withdraw the FlashCopy at the remote
site.
This eliminates the need for the network connection to the remote site solely for FlashCopy
management. Remember that you still need the network connection to the remote site for the
purpose of creating the inband FlashCopy task.
Note: Remember that while accessing the target volume (even in read only mode), the
operating system might need to update some piece of data on the target volume. So if
writes are inhibited to the target volume while accessing it, it might produce unpredictable
results depending on the volume manager or operating system used.
Note: The background copy process must be finished prior to using this option.
Note: The Inband command can also be used with the FlashCopy Start Background
Copy option.

Chapter 3. FlashCopy
45
3.4.5 Consistency Created task with FlashCopy V2
In addition to the Establish FlashCopy pair and Withdraw FlashCopy pair tasks, there is a
new type of task called Consistency Created, which is issued on the LSS level (from the
Logical Subsystems panel in the ESS Copy Services Web User Interface (WUI)).
For FlashCopy, it is created with the following option.
FlashCopy Consistency Group (One LSS Selected)
This option is used to reset the Queue Full condition after the Establish FlashCopy pair task
with the Freeze FlashCopy Consistency Group option was run. All write operations to the
source volume(s) that were queued by the host are written as soon as this task completes.
See 6.11.10, “Thawing a FlashCopy Consistency Group” on page 255 to learn how to create
the task. For more information about FlashCopy Consistency Groups, see 3.3.4, “FlashCopy
Consistency Groups” on page 39.
3.4.6 FlashCopy V2 enhancements (LIC 2.4.x)
With the ESSLIC level 2.4.x or higher, the support for the Asynchronous PPRC was
introduced. There are also new options for the FlashCopy V2 Copy Services function, which
are used mainly in the Asynchronous PPRC environment (for more information about
Asynchronous PPRC, refer to 4.5, “Asynchronous PPRC” on page 74). Below you can see
the new options available with ESS LIC level 2.4.x or higher for FlashCopy V2:
Establish FlashCopy pair task options:
Fast Reverse Restore
Establish FlashCopy target on existing PPRC Primary
Revertible
Withdraw FlashCopy pair task options:
Revert the FlashCopy
Commit FlashCopy
There is also new status information for the FlashCopy volume pairs:
FlashCopy sequence number: This is a unique number which is assigned to all FlashCopy
pairs by the Asynchronous PPRC mechanism every time a new Consistency Group is
created successfully.
Revertible: This is a status flag for the FlashCopy volume pair that tells you if you can
revert the FlashCopy pair to the previous FlashCopy sequence.
These options, except for Establish FlashCopy target on existing PPRC Primary, are
used mainly in an Asynchronous PPRC environment and should not be used for other
purposes (for more information on these options, please refer to 4.5, “Asynchronous PPRC”
on page 74).
Establish FlashCopy target on existing PPRC primary
With the ESS LIC level prior to 2.4.x, it was possible to establish a FlashCopy relationship
between two volumes and then establish a PPRC copy pair by using the existing FlashCopy
target as PPRC primary volume. With LIC level 2.4.x or higher, it is possible to use an existing
Note: Because this task is created on the LSS level, it is run on all the FlashCopy pairs that
are in the Queue Full condition within the selected LSS.

46
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
PPRC primary volume as a FlashCopy target. This ability can enable the implementation of a
multi-hop remote copy solution as an alternative to other vendors solutions.
The FlashCopy to an existing PPRC primary volume can be a full or incremental point-in-time
copy. FlashCopy Consistency Groups can be used, the No background copy option can
be specified and the relationship may be persistent. Data set level FlashCopy in z/OS
Systems is also permitted. PPRC can run either in Synchronous or in XD Mode. The PPRC
primary may also be a Synchronous PPRC suspended primary. The only exception is that
you cannot use the Asynchronous PPRC with this new function at the primary site.
Figure 3-5 gives an overview of the FlashCopy to a PPRC primary volume process.
Figure 3-5 FlashCopy to a PPRC primary volume
When the FlashCopy task is run, the corresponding PPRC volume pairs turn to copy pending
status as long as the data is being transferred to the PPRC secondary site. At this time the
secondary site will be inconsistent, if a disaster occurs.
3.5 Planning for FlashCopy
Because FlashCopy invariably will be used on production systems, you should carefully plan
the setup of your environment and test it thoroughly. This is an important step to minimize the
possibility of error and potential rework.
PPRC
FlashCopy to a PPRC primary volume
PPRC
Local Site Remote Site
FlashCopy
- full volume / incremental
- nocopy
- persistent / non-persistent
- data set level FC
no Asynchronous PPRC
possible!
PPRC Primary
FlashCopy Source
PPRC Primary
FlashCopy Target
FlashCopy Target PPRC pair in copy pending mode until entire copy complete
B'
A'
B
A

Chapter 3. FlashCopy
47
3.5.1 Hardware and software requirements
If you want to use FlashCopy, you need to comply with the following prerequisites:
Have a FlashCopy feature purchased and enabled on your Enterprise Storage Server
(ESS) by the Customer Engineer (CE). The feature code is dependent on the total disk
capacity of your ESS, rather than on the capacity of the volumes that will use FlashCopy. It
is also dependent on whether you want to use FlashCopy V1 or FlashCopy V2. Refer to
Appendix B, “Copy Services feature codes” on page 541 for the feature code information.
On the server that will have the FlashCopy target volumes attached, you need to have
enough SCSI target IDs and/or SCSI/Fibre Channel LUNs available (not occupied by
volumes). The ESS can have up to 15 SCSI target IDs each with up to 64 LUNs on one
SCSI channel and up to 4095 LUNs on a Fibre Channel port.
You need TCP/IP connectivity between the ESS and the host system that will initiate
FlashCopy (usually this is the system that will access the FlashCopy target) in order to use
the Copy Services Command Line Interface (CLI). You can achieve that by connecting the
ESS to the company intranet. You have to install the CLI on the host that will be using it.
If you have Independent Software Vendor (ISV) software installed, you need to contact the
ISV regarding their support for ESS Copy Services.
Review your volume manager software considerations for FlashCopy (refer to Chapter 8,
“Open systems specifics” on page 407):
– AIX LVM
– VERITAS VxVM
– HP SAM
3.5.2 Configuration planning
The most important consideration that has to be taken into account while using FlashCopy V1
is to have an available volume (LUN) in the logical subsystem (LSS) where the source
volume resides. With FlashCopy V2, you can use any LUN within a single ESS as a target.
The target LUN has to be of the same size as the source or bigger. The space for target data
has to be available even if only the No background copy option will be used.
For an understanding of LSS concepts, see Appendix C, “Logical subsystems” on page 549.
3.5.3 Resource planning
When planning your ESS volume layout, it is important to consider the capacity you may need
for FlashCopy targets. Bear in mind that the disk space you need is real disk space. For
FlashCopy V1, you must also consider that the target volume is restricted to the same LSS as
its source volume. So, when you allocate additional storage for FlashCopy targets, consider
how much space in each LSS you need to leave unallocated.
With Version 1, you cannot initiate a FlashCopy session on a source and target that are
already in a FlashCopy session. You need to wait for the FlashCopy task to complete or you
can withdraw the pair manually. If you have used the No background copy option, you
always need to withdraw the pair (unless all tracks on the source volume have changed).
Note: If using FlashCopy V1, you may need to review your configuration to have a target
volume available in the source LSS. That will usually mean copying some of your data to
another LSS, using either the host system or the PPRC links.

48
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
3.5.4 Data consistency considerations
It is very important to verify that the copy of the data you will be using is fully consistent using
a proper file system check procedure provided by your operating system. If you are going to
automate your FlashCopy procedures, consider including this check each time when you
make the FlashCopy target available to the host. In all cases before starting the FlashCopy
procedure, the target volume must be unmounted; this is to ensure that there is no data in any
system buffers that can be flushed to the target and could potentially corrupt it.
3.5.5 Test plan and Disaster Recovery plan
If you plan to use FlashCopy, you need to test your setup. Do not forget that you are dealing
with a binary copy of the data that was done out of the control of your operating system.
Prepare a test plan and, if you are using FlashCopy for backup/restore, a recovery plan.
3.6 Operational considerations
The following sections are intended to help you manage your FlashCopy pairs and, in
particular, to manage the target you create.
3.6.1 Monitoring and managing FlashCopy pairs and volumes
FlashCopy pairs and tasks can be managed by both the ESS Copy Services Web User
Interface (WUI) and the Command Line Interface (CLI) on the host.
The ESS Copy Services Web User Interface (WUI) will allow you to manage FlashCopy
volumes and tasks (for more information, see 5.10, “Performing FlashCopy with the Web User
Interface” on page 150 and 6.11, “Performing FlashCopy operations with the WUI” on
page 225). You can establish and withdraw a FlashCopy by clicking on the graphical
representations of the volumes in the WUI. If you wish to perform a FlashCopy from the CLI
(for more information, see Chapter 7, “ESS Command Line Interface” on page 377), you must
create a FlashCopy task within the WUI and save it. You can either execute your tasks from
the WUI or call them with the rsExecuteTask.sh command in the CLI.
Using the CLI with predefined tasks minimizes the danger of a human error when handling
physical volumes by their volume numbers or names from the ESS Copy Services Web User
Interface, and it enables automation.
3.6.2 Using a FlashCopy target volume
Remember that if you have established a FlashCopy with the No background copy option,
you need to withdraw the FlashCopy pair after you have finished using the FlashCopy target
volume. If you choose to perform a full copy, the relationship will be withdrawn automatically
when the background copy task ends. The performance issues of using FlashCopy with full
copy and No background copy options are discussed in 3.7, “Performance considerations”
on page 49.
If you will be using FlashCopy for data backup purposes, change your recovery procedure so
that you will be able to recover even when the data has been backed up by a different backup
client than the original owner of the LUN, or it has been backed up from a different location in
the file system (the target mount point).
You can, of course, perform the FlashCopy from the target volume to the original LUN using
the full copy option (but only
after
the background copy has completed).

Chapter 3. FlashCopy
49
3.6.3 Automation
Different operating systems allow different levels of automation. The automation can be done
using batch or script files. This is useful, for example, to create an automatic daily backup
procedure.
In the script file that you use for creating the target volume(s), you should include:
1.Quiescing of an application (switching on the backup mode). The proper quiesce
procedure is provided in your application’s documentation.
2.Flushing data to the source volumes. This can be accomplished by unmounting the source
volumes, but sometimes it may be necessary to shut down the source server.
3.Unmounting the FlashCopy target volumes from the target system (if they were already
used).
4.Establishment of FlashCopy pair(s) using the Copy Services Command Line Interface with
the rsExecuteTask.sh command.
5.Resuming an application (terminating the backup mode) using the procedure described in
your application’s documentation.
6.Hardware scan for new disks on a target system (in case their definitions are not already
present).
Optional steps:
7.Verifying the consistency of FlashCopy target volumes.
8.Mounting the target volumes on a target system.
After you finish using the target volume (for example, the backup to tape has completed), you
should run a script including the following steps:
Unmounting the target volumes from a target system (if you did not do this before running
the FlashCopy).
Withdrawal of the FlashCopy pair (in case the No background copy option was used)
with rsExecuteTask.sh.
For detailed information about automation on different operating systems, see Chapter 8,
“Open systems specifics” on page 407.
3.7 Performance considerations
The next section is intended to give you the considerations involved when setting up the Copy
Services of the Enterprise Storage Server (ESS) in order to achieve better performance.
This should help you understand the performance impact of ESS Copy Services. As there are
many different parameters that have an influence on performance, such as applications, kind
of workload, and the configuration of the Enterprise Storage Server, this information should
serve as a guideline when planning for ESS Copy Services.
Note: Do not attempt to defragment or optimize a FlashCopy source volume while the
FlashCopy background copy task is running. This can significantly degrade the
performance.

50
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Keep in mind that the general ESS performance considerations, such as the volume
placement or amount of storage per host adapter, still apply when planning for FlashCopy.
3.7.1 Placement of source and target volume
As we discussed earlier in this book, with FlashCopy V1, the source and target volume of a
FlashCopy pair must be in the same logical subsystem (LSS). In certain configurations of the
ESS, a single LSS includes more than one RAID 5 or RAID 10 Array (rank). In such
configurations, we recommend that you use a target and source volume from different ranks
to make your FlashCopy pair. Furthermore, we recommend that you use ranks for your
source and target volumes that reside on different SSA loops of the Device Adapter (DA) pair,
if this is possible in your ESS configuration (in open-systems environment, this is only
possible if the ESS is set up to use eight Fixed Block logical subsystems). This will distribute
the I/O load over more disks, as data is copied from the source to the target volume.
With FlashCopy V2, we recommend that you use different logical subsystems for the source
and the target volumes, so the I/O operations are distributed over different SSA Device
Adapters.
If you are making a FlashCopy with the Full copy option when all data from the source is
physically copied to the target (default), and you do not have to work with the target volume
directly after the FlashCopy was issued, we recommend waiting until the background copy is
finished. This will give you better performance for both the source and the target volume, as
host I/O requests will not interfere with the I/O of the FlashCopy task. The progress of the
FlashCopy background copy process can be determined with the Copy Services Web User
Interface and the completion of the process by using the CLI.
3.7.2 No background copy option
Consider whether you want to select the No background copy option for FlashCopy or not. If
you have mainly read access to the source and the target of your FlashCopy pair, we
recommend using the No background copy option to minimize I/O traffic to the RAID arrays.
Keep in mind that when selecting the No background copy option, the relationship between
the FlashCopy source and target stays until the pair is withdrawn manually (unless all tracks
on the source volume have been changed).
3.7.3 Number of simultaneous FlashCopy pairs
Consider the number of FlashCopy pairs you have active at the same time. The time for a
single FlashCopy pair establish and the copy time to finish will increase with the amount of
FlashCopy pairs you have established at the same time as data is copied in between all pairs.
Try to logically group FlashCopy pairs together for ease of use.
3.8 Practical examples using FlashCopy
In this section, we discuss various practical examples of using FlashCopy.
3.8.1 Moving and migrating data
Any time you need to move data from one server to another, FlashCopy can be useful. Do not
forget to quiesce disk access before making a FlashCopy and verifying the consistency of the
data before attaching it to the target server.

Chapter 3. FlashCopy
51
3.8.2 Moving workload
The same way you move the data from one server to another, you can move workload
between servers. To move workload, you can usually only reassign the volume in the ESS
Specialist. For details, refer to the IBM TotalStorage Enterprise Storage Server Web Interface
User’s Guide, SC26-7448, or for more information on the ESS Storage Management CLI, see
Chapter 7, “ESS Command Line Interface” on page 377.
3.8.3 Backup
FlashCopy does not usually speed up your physical backup, but it allows you to run your
application while you do the backup; therefore, the physical backup completion time becomes
less important for you. You may need fewer tape drives and a less powerful backup server.
You need to shut down your application only for the time the FlashCopy task is started and
can restart it almost immediately when the task is completed.
While using FlashCopy V1, the performance of the LSS in which the FlashCopy pair resides
may be reduced by a very small percentage during the background copy process.
You can keep FlashCopy targets online after you back them up, so you will be able to copy the
files that need to be restored from the FlashCopy target rather than having to restore from
tape.
FlashCopy also enables you to do backups whenever you want, not only during the off-shift
period, because you do not need to wait for a reduced server load in order to do the backup.
You can use all your existing backup software to do backups. However, in case you intend to
do full and incremental (differential) backups, you need to check how your software records
the files that have been backed up. In some cases, it marks the files as
archived
on the target
disk. This change, of course, is not reflected in the FlashCopy source volume, so anytime you
withdraw the FlashCopy target and you attempt incremental backup on the next FlashCopy of
that source, you back up all the files again.
There are three methods you can use to restore the data when the target is still available:
1.Mapping the target to the source host and the original mountpoint/drive letter using the
ESS Specialist or the ESS Storage Management CLI.
2.Copying the data back to the source either using the standard operating system means or
doing a reverse FlashCopy, that is, from the original target back to the source. Note that
you can only do this if the full background copy has completed.
3.Creating a new FlashCopy of the target and assigning it to the source host.
Database offline backup
The scenario shown in Figure 3-6 on page 52 provides very fast backups and restores of a
database. Logically, the data is being copied from copy (A) to copy (B).
Attention: Be careful if you have the two volumes online to the same host with the same
volume IDs.

52
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 3-6 Database Backup scenario
The steps to perform the backup are as follows:
If application consistency is required, include these first two steps. For a discussion of
consistency, please go to 9.2, “Data consistency” on page 458.
1.Stop production services and the database server.
2.Flush the file system buffer on the production server and secondary server.
Then continue with these steps:
3.Start your FlashCopy task with the WUI or the CLI.
4.As soon as the relationship has been established, restart production services and the
database server.
5.Refresh the secondary file system (make the secondary server aware of the new disks if
needed).
6.Attach the database on the secondary site.
7.Perform database integrity checks at the secondary site.
8.Start production services on the secondary site (optional).
Database online backups
If you require 24x7 availability, you can use the scenario shown in Figure 3-6 to perform an
online backup instead of an offline backup of your database. While the backup is not
necessarily in a known state, it is still consistent and can be used for recovery. In general, the
online backup is done by quiescing the data base on the production server, performing the
FlashCopy and then restarting the data base. You can accomplish this by using the supported
backup features of your database. For further information, see your database documentation
or refer to the ESS whitepapers on the following Web site:
http://www.storage.ibm.com/disk/ess/whitepaper.htm
ESS
Secondary Server
Primary Server
B
A
FlashCopy
FlashCopy
hCopy

Chapter 3. FlashCopy
53
3.8.4 Application testing
You can test new applications and new operating system releases against a FlashCopy of
your production data. Therefore, you can eliminate the risk of data corruption, and your
application does not need to be taken offline for an extended period of time in order to
perform the copy of the data.
3.8.5 Other examples
Data mining is a good example of where FlashCopy can help you. Data mining can now
extract data without affecting your application.
In the following sections, we give examples for the usage of FlashCopy and PPRC. The
examples are intended to show solutions that are possible with ESS Copy Services. They are
not related to a specific operating system and could be used on all supported operating
systems.
Moving from a single-host environment to a clustered environment
In a clustered environment, two or even more servers are accessing the same resources,
such as disk drives. There are different clustering models available that provide the following
benefits. A cluster increases the availability of the data, as the control of the resources from a
server that is not available anymore is transferred to the remaining server(s). The failover,
failback, and access to the resources is controlled by special cluster software running on all
servers within a cluster. Depending on the clustering model, the workload may also be shared
between the nodes within a cluster.
If you are currently running your applications in a non-clustered environment and you are
planning to move to a clustered-environment, ESS FlashCopy functionality can be used to
create a test environment for the cluster very quickly and easily with minimal impact on
production.
In order to test the cluster, you need to prepare the new host systems of the cluster in
advance. This is needed to create an identical environment from the host point of view.
Issue the FlashCopy command on all the volumes you want to use in the cluster environment
later on. The FlashCopy target volumes are shared with the new hosts of the cluster. Disk
sharing with the Enterprise Storage Server is very easy, as the FlashCopy target LUNs only
need to be assigned to multiple host adapters. This is done with the ESS Specialist Web User
Interface. It is also possible to move to Fibre Channel connectivity in the clustered
environment if the shared target volumes are connected to a Fibre Channel host port.
Once the FlashCopy pair is established, the shared target volumes with the time-zero copy of
the production data could be tested in the clustered environment. However, we recommend
waiting until the data copy from the source to the target is finished. This will ensure that there
is minimal impact on production I/O when testing on the FlashCopy targets.
This example is illustrated in Figure 3-7 on page 54.

54
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 3-7 Moving cluster environment using FlashCopy
Testing a cluster environment
If there is already a clustered environment, you can use FlashCopy to duplicate the shared
data on the target volume(s). In this case, both the FlashCopy source and target volumes are
LUNs that are shared on the ESS. This may be applicable if you want to test your cluster
without having any impact on the production environment. Refer to Chapter 8, “Open systems
specifics” on page 407 for information on the issue of having duplicate vol IDs online to the
same host.
Data backup with minimal impact on production
ESS FlashCopy functionality can be used to do online backups of production data. As the
data on the FlashCopy target is immediately available after the FlashCopy relationship is
established, you need to ensure a consistent state of the data on the source volume for a few
seconds only. After the short time of FlashCopy establishment, the data on the target could
be written to the backup media immediately. This decreases the time of backup windows of
production data significantly.
Non-cluster
environment
Cluster environment
Enterprise Storage Server
FlashCopy
source
FlashCopy
target
(shared volume)
FlashCopy
Servers

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
55
Chapter 4.
Peer-to-Peer Remote Copy
(PPRC)
In this chapter, the characteristics and operation of ESS Copy Services Peer-to-Peer Remote
Copy (PPRC) are described. This chapter also discusses useful considerations when
planning the implementation of PPRC.
PPRC Version 1
– Synchronous PPRC, also known as IBM TotalStorage Metro Mirror
– PPRC-XD, also known as IBM TotalStorage Global Copy
PPRC Version 2
– Asynchronous Cascading PPRC, also known as IBM TotalStorage Metro/Global Copy
– Asynchronous PPRC, also known as IBM TotalStorage Global Mirror
PPRC Pathing
– ESCON
– PPRC over FCP links
Planning for PPRC
Practical uses of PPRC
This chapter helps you to understand how PPRC works and to plan for its usage. Chapter 5,
“ESS Copy Services Web User Interface prior to LIC 2.2.0” on page 123, Chapter 6, “ESS
Copy Services Web User Interface: LIC 2.2.0 and later” on page 189, and Chapter 7, “ESS
Command Line Interface” on page 377 show you how to set up PPRC step by step. PPRC is
also covered in Chapter 9, “Disaster Recovery” on page 453.
4

56
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.1 PPRC overview
PPRC is a function of a storage server that constantly updates a secondary copy of a volume
to match changes made to a primary volume. The primary and the secondary volumes can be
on the same storage server or on separate storage servers.
PPRC differs from FlashCopy in two essential ways. First, as the name implies, the primary
and secondary volumes can be located at some distance from each other. Second, and more
significantly, PPRC is not aimed at capturing the state of the source at some point in time, but
rather aims at reflecting all changes made to the source data at the target.
The ESS implementation of PPRC provides three modes of operation:
Synchronous PPRC Maintains a consistent copy. All modifications that any attached
host performs on the primary volume are also performed on the
secondary logical volume before operation completion is
indicated.
PPRC Extended Distance Maintains a fuzzy copy. All modifications that any attached host
performs on the primary volume are also performed on the
secondary volume at a later point in time. The original order of
updates is not strictly maintained.
Asynchronous PPRC Based on PPRC-XD functionality, it creates a consistent copy of
data at the remote site periodically using Inband FlashCopy.
Operating over high-speed Fibre Channel communication links, it
is designed for virtually no distance limitation and provides data
consistency at the remote site with no significant impact on the
application performance.
There is also Asynchronous Cascading PPRC, which in fact is not a mode of PPRC
operation, but it is a combination of Synchronous PPRC and PPRC-XD modes. It is mainly
(but not necessarily) used in three-site disaster recovery solutions.
PPRC is application independent. Because the copying function occurs at the disk subsystem
level, the host’s operating system or application has no knowledge of its existence. In contrast
to that, host-based mirroring is controlled by software at the operating system or file system
level: The storage subsystem does not know about that. Table 4-1 summarizes characteristics
of both approaches.
Table 4-1 Comparison of PPRC and host-based mirroring
Peer-to-Peer Remote Copy Host-based mirroring
Operation is performed by the storage
subsystem, and is transparent for the host
operating system. The functionality is the same
for all operating systems and applications.
Operation is performed by the host software or
host bus adapter, and is transparent for the
storage subsystem. The functionality depends on
capabilities of the operating system or host bus
adapter.
Read and write operations are sent to the primary
volume only.
Write operations are sent to both volumes. Read
operations are sent to any volume, depending on
read policy.
There is an unidirectional relationship from the
primary to the secondary volume. Failure
recovery is different for the primary and
secondary volume.
The relationship between the volumes is
symmetric. Failure recovery is identical for both
volumes.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
57
4.2 Synchronous PPRC
PPRC can be viewed as having two phases: The first is an initial copy phase involving a bulk
transfer of all of the data at the primary site to the secondary site. Large amounts of data are
copied in large units. During a second phase, modifications are transferred that occur at the
primary site; this may include updates that occurred during the first phase. This continuous
phase involves transferring smaller units of data, according to the I/O size of the applications.
4.2.1 Synchronous PPRC initial copy
When a PPRC copy pair relationship has been initially established or re-established after
being suspended, the volumes are in a
duplex pending
state (see 4.2.3, “Synchronous PPRC
volume states” on page 58). While in this state, PPRC maintains a bitmap to keep a record of
updated tracks. On an initial copy pair establish, PPRC will parse the volume copying of all
the tracks (as if all the bits are set). On the second parse of the volume, PPRC will only copy
tracks that were changed since the last parse.
While in this state, the PPRC algorithm will signal a write complete to the application server
as soon as the write I/O has been committed to the primary ESS cache/NVS structure. Once
the primary and secondary volumes have been synchronized, the volume state changes to
duplex
and the PPRC algorithm now signals the completion of a write cycle, once it receives
acknowledgement from the secondary ESS that the data has been committed to its cache
and NVS. The sequence of events is illustrated in Figure 4-1. (The icons are from the ESS
with LIC prior to 2.2.x installed; to see icons for ESS with LIC 2.2.x or higher, refer to
Chapter 6, “ESS Copy Services Web User Interface: LIC 2.2.0 and later” on page 189.)
Figure 4-1 PPRC initial volume synchronization
Secondary
Volume
All tracks
copied
0
0
1
0
1
0
0
0
1
0
0
0
0
1
1-a.At initial copy, PPRC does
a pass across the volume,
copying all the tracks.
3.Now the volume pair is
in full duplex mode and
all the write updates are
mirrored synchronously.
0
0
1
0
1
0
0
0
1
0
0
0
0
1
2-b.When this second step is started, and from then on, any write update on an
already copied track is mirrored synchronously as when in full duplex mode.
1-b.A bitmap is kept to check for already copied tracks that receive an update
before the initial copy pass is finished.
Primary
Volume
0
0
1
0
1
0
0
0
1
0
0
0
0
1
0
0
1
0
1
0
0
0
1
0
0
0
0
1
Only changed
tracks copied
Updates are
transferred
synchronously
2-a.A second pass is done
copying just the updated
tracks that were checked
in the bitmap.

58
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.2.2 Synchronous PPRC continuous operation
The Synchronous PPRC protocol guarantees data consistency by ensuring that the
write
complete
is only received by the host application once the remote copy has been committed
to the secondary ESS and acknowledged by both ESSs. This is illustrated in Figure 4-2.
Figure 4-2 PPRC write I/O cycle
As shown in Figure 4-2:
1.The host server requests a write I/O to the primary ESS. The write is staged through
cache and non-volatile storage (NVS).
2.PPRC dispatches the write over an ESCON or FCP link to the secondary ESS. The write
hits the secondary ESS’s cache and NVS, which is sufficient to initiate an
acknowledgement of the remote write to the primary ESS.
3.The primary ESS waits for acknowledgment of the remote write. If the secondary write
fails, the acknowledgement does not return to the host server, causing an I/O timeout to
the host server, which in turn causes a retry from the host server.
4.The write returns to the host server’s application.
Once acknowledgement of the write has been received by the primary, both the primary and
secondary write I/Os are eligible for destaging to disk. Destaging from the cache to the disk
drives on both the primary and the secondary ESS is performed asynchronously.
If acknowledgement of the remote write is not received within a fixed period of time, the write
is considered to have failed, and is rendered ineligible for destaging to disk. At this point, the
application receives an I/O error, and in due course, the failed write I/O is
aged-out
of each
NVS.
4.2.3 Synchronous PPRC volume states
Volumes within the Enterprise Storage Server used for Synchronous PPRC can be found in
one of the states illustrated in Figure 4-3 on page 59. (The icons are from the ESS with LIC
prior to 2.2.x installed; to see icons for ESS with LIC 2.2.x or higher, refer to Chapter 6, “ESS
Copy Services Web User Interface: LIC 2.2.0 and later” on page 189.)
2
3
1
4
Server write
Write to secondary
Write acknowledge
Write hit
to secondary

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
59
Figure 4-3 PPRC volume states
Simplex: The simplex state is the initial state of the volumes before they are in a PPRC
volume pair relationship, or after the PPRC relationship has been terminated. Both
volumes are accessible only when in this state.
Duplex pending: Volumes are in duplex pending state after the PPRC copy relationship is
established, but the source and target volume are out-of-sync. In that case, data still
needs to be copied from the source to the target volume of a PPRC pair. That may be the
case either after the PPRC relationship was just established (or re-established after being
suspended), or in the case where the PPRC volume pair re-establishes after a storage
subsystem failure. The PPRC secondary volume is not accessible when the pair is in
duplex pending state.
Duplex: The duplex state is the state of a volume copy pair whose members are in sync,
that is, both source and target volumes contain exactly the same data. This state is also
referred to as
full copy mode
. The PPRC secondary volume is not accessible when the
pair is in duplex state (if the Permit read from secondary option was not activated).
Suspended: Volumes are in the suspended state when the source and target storage
subsystems cannot communicate anymore, or when the PPRC pair is suspended
manually. In this state, writes to the primary volume are not mirrored onto the secondary
volume. The secondary volume becomes out-of-sync. During this time, PPRC keeps a
bitmap record of the changed tracks in the primary volume. Later, when the volumes are
re-synchronized, only the tracks that were updated will be copied.
Primary
Volumes
Secondary
Volumes
Primary Site
Secondary Site
Volume State
Synchronization
Duplex
Simplex
Suspended
Duplex
Duplex
Pending
Data Copy
PPRC pair is
suspended,
volumes are no
longer in sync
Volumes have no
PPRC relationship
PPRC pair is
established, with
volumes in full
synchronization
Not in sync, but
the PPRC pair is
established
0
0
1
0
1
0
0
0
1
0
0
0
0
1

60
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.2.4 Synchronous PPRC tasks and options
There are three types of ESS Copy Services tasks to establish, suspend, and terminate
Synchronous PPRC. Each of these tasks can be used with several options.
Establish Synchronous PPRC copy pair: This operation starts the transition of two simplex
volumes into a pair in the duplex pending or duplex state. You have to select one out of the
four options below to specify the amount of data to be transferred during initialization:
Copy entire volume copies all tracks from the primary volume to the secondary,
overwriting the whole secondary volume. The volume pair changes from state simplex to
duplex pending and finally to duplex after the copy is completed.
Do not copy volume establishes the PPRC pair without any data transfer. The volume
pair state goes to duplex immediately. This option presumes that the volumes are already
synchronized. It allows you to restart a PPRC operation in the shortest time possible when
you are sure that there were no data modifications.
Copy out-of-sync cylinders only requires that the volumes are in the suspended state
with a bitmap record of the changed tracks in the primary volume. The volume pair state
changes from suspended to duplex pending and finally to duplex after the copy is
completed. You choose this option to restart a PPRC operation when there are data
modifications on the primary volume.
PPRC Failover and PPRC Failback reverse the primary and secondary roles during a
failover or failback process. The amount of data to be transferred depends on the volumes’
state. We will discuss this in Chapter 9, “Disaster Recovery” on page 453.
The following additional options can be selected:
Permit read from secondary allows host servers to read from the PPRC secondary
volume. The PPRC pair must be in a duplex state in order for the host server to read the
volume. This option is helpful for operating systems with a Logical Volume Manager that
reads data structures from the header of a secondary volume to update internal
databases.
Suspend PPRC after establish complete suspends the PPRC pair immediately after the
establish-pairs command completes. This option is useful in combination with PPRC-XD
(see 4.3, “PPRC Extended Distance (PPRC-XD)” on page 61).
Asynchronous Cascading PPRC will be discussed in 4.4, “Asynchronous Cascading
PPRC” on page 67.
Please note that the options Critical volume mode and Permit establish if target is online
are not available for open-systems volumes.
Suspend PPRC copy pair: This operation stops transferring data to the target volume.
Because the primary ESS keeps track of all changed cylinders on the source volume, you can
resume PPRC operations at a later time. You have to select one of the two options:
Schedule task with source logical subsystem is the default option. Both volumes will
go to the suspended state.
Schedule task with target logical subsystem may be necessary when the primary LSS
is not available. The primary volume might appear later still in state duplex, not creating a
bit map record of changed tracks.
Terminate PPRC copy pair: This operation ends the relationship between the volumes.
Similar to Suspend PPRC copy pair, you have to select one of the two options:
Schedule task with source logical subsystem is the default option. Both volumes will
go to state simplex.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
61
Schedule task with target logical subsystem may be necessary when the primary LSS
is not available. The secondary volume will go to the simplex state, the primary volume will
be in a suspended state.
Operations and options will be discussed again with respect to task creation in Chapter 6,
“ESS Copy Services Web User Interface: LIC 2.2.0 and later” on page 189 and Chapter 9,
“Disaster Recovery” on page 453.
4.3 PPRC Extended Distance (PPRC-XD)
In this mode of operation, PPRC copies data from the primary volume to the secondary
volume in a
non-synchronous
manner. This means that I/O write completions are returned to
the application once they have been committed to the primary ESS. Updates on the
secondary volume are performed at a later point in time. The original order of updates is not
strictly maintained.
The non-synchronous characteristics of PPRC-XD, combined with the throughput of the
efficient track mirroring technique, and with enhanced microcode support for channel
extenders, makes PPRC-XD well suited for remote copy solutions at distances beyond
103 km (300 km with PPRC over Fibre Channel), without having to incur distance latency
penalties that synchronous write I/O cycles exhibit. Of course, it may be used in environments
where the distance is less than 103 km (or 300 km with PPRC over Fibre Channel) if the
performance impact of synchronous processing is not admissible. This behavior means that
distance does not directly impact the performance of the application.
4.3.1 PPRC-XD operation and volume states
When operating in PPRC-XD mode, the volume pairs are in the
duplex pending XD
state.
This state is in addition to those already described in 4.2.3, “Synchronous PPRC volume
states” on page 58. (The icons in Figure 4-4 are from the ESS Copy Services panels (ESS
LIC prior to 2.2.x); to see icons for ESS with LIC 2.2.x or higher, refer to Chapter 6, “ESS
Copy Services Web User Interface: LIC 2.2.0 and later” on page 189.)
Figure 4-4 PPRC Extended Distance duplex pending XD volume state
While the volume pairs are in the duplex pending XD state, PPRC is doing a
non-synchronous mirroring of the primary volumes’ updates on to the secondary volumes.
PPRC-XD maintains a bitmap of the changed tracks for all the primary volumes in the duplex
pending XD state. It periodically cycles through each volume’s bitmap for updated tracks and
then sends the updates in a batch of 30 tracks for transmission to their secondary
Primary Site
Secondary Site
Primary
Volumes
Secondary
Volumes
Volume State
Synchronization
and
consistency
Duplex
Pending XD
Not in sync
and
fuzzy
0
0
1
0
1
0
0
0
1
0
0
0
0
1
non-sychronous
update

62
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
counterparts. Once the batch has been committed to the cache/NVS of the secondary ESS
and the original track bit has not been set again, the bit is cleared to avoid unnecessary
retransmissions. This is a very efficient throughput-oriented method of non-synchronous
mirroring.
The efficient extended distance technique of PPRC Extended Distance is achieved using
sophisticated algorithms. For example, if changed data is in the cache, then PPRC sends only
the changed sectors. There are also sophisticated queueing algorithms to schedule each
volume for processing of their updated tracks. These PPRC-XD operational details are
beyond the scope of this document.
The volume pair will remain in the duplex pending XD condition, not reaching the duplex state
until instructed to do so. The operation to exit the duplex pending XD state will direct the
volume pair either to go to the duplex state (a
go-to-sync
operation), or to the suspended
state (a suspend pair operation), or to the simplex state (a terminate pair operation). The
permissible volume state transitions are illustrated in Figure 4-5. (The icons are from the ESS
Copy Services panels (LIC level prior to 2.2.x); to see icons for ESS with LIC 2.2.x or higher,
refer to 6.5.1, “Volumes panel icons” on page 201.)
Figure 4-5 PPRC-XD volume state transitions
4.3.2 PPRC-XD tasks and options
Similar to Synchronous PPRC, there are three types of ESS Copy Services tasks to establish,
suspend, and terminate PPRC-XD with several options for each tasks.
suspended simplexfull-duplex
changed tracks
updates only
0
0
1
0
1
0
0
0
1
0
0
0
0
1
suspendedsimplex
s
u
s
p
e
n
d

p
a
i
r
c
a
t
c
h
-
u
p
g
o
-
t
o
-
s
y
n
c
d
e
l
e
t
e

p
a
i
r
inital copy
XD option
incremental
resync
XD option
PPRC-XD periodically cycles
thru the volumes for updated
tracks.
Volumes remain in duplex
pending XD state until
commanded to exit.
New option XD when
doing an initial copy or
when doing an
incremental
resynchronization.
Volumes are scheduled for
subsequent transfer of updates
to secondaries.
non-synchronous
Non-synchronous mirroring
frees application writes
from synchronous overheads.
Secondary fuzzy copy is
made consistent with
adequate catch-up.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
63
Establish PPRC Extended Distance copy pair starts the transition of two simplex volumes
into a pair in a duplex pending or duplex state. You have to select one of the four options
below to specify the amount of data to be transferred during initialization:
Copy entire volume copies all tracks from the primary volume to the secondary,
overwriting the whole secondary volume.
Do not copy volume establishes the PPRC pair without any data transfer. This option
presumes that the volumes are already synchronized. It allows you to restart a PPRC
operation in the shortest time when you are sure that there were not any data
modifications (this option is not available for PPRC-XD with ESSs with LIC prior to 2.2.x
installed).
Copy out-of-sync cylinders only requires that the volumes are in a suspended state with
a bitmap record of the changed tracks in the primary volume. You choose this option to
restart a PPRC operation when there were data modifications on the primary volume.
PPRC Failover and PPRC Failback reverse the primary and secondary roles during a
failover or failback process (PPRC-XD Failover is supported with ESSs with LIC 2.3.x or
higher installed). The amount of data to be transferred depends on the volumes’ state. We
will discuss this in Chapter 9, “Disaster Recovery” on page 453.
The following additional options can be selected:
Permit read from secondary allows host servers to read from the PPRC-XD secondary
volume. Unlike a Synchronous PPRC pair, the pair is not required to be in a duplex state
for the host server to read the secondary volume (refer to Chapter 8, “Open systems
specifics” on page 407).
Asynchronous Cascading PPRC will be discussed in 4.4, “Asynchronous Cascading
PPRC” on page 67.
Please note that the options Critical volume mode and Permit establish if target is online
are not available for open-systems volumes. The option Suspend PPRC after establish
complete is available only with Synchronous PPRC.
Suspend PPRC pair: This operation stops transferring data to the target volume. Because
the primary ESS keeps track of all changed cylinders on the source volume, you can resume
PPRC operations at a later time. You have to select one of the two options:
Schedule task with source logical subsystem is the default option. Both volumes will
go to the suspended state.
Schedule task with target logical subsystem may be necessary when the primary LSS
is not available. The primary volume might appear later still in duplex state, not creating a
bit map record of changed tracks.
Terminate PPRC pair: This operation ends the relationship between the volumes. Similar to
Suspend PPRC pair, you have to select one of the two options:
Schedule task with source logical subsystem is the default option. Both volumes will
go to state simplex.
Schedule task with target logical subsystem may be necessary when the primary LSS
is not available. The secondary volume will go to the simplex state, and the primary
volume will be in the suspended state.
Operations and options will be discussed again with respect to task creation in Chapter 6,
“ESS Copy Services Web User Interface: LIC 2.2.0 and later” on page 189 and Chapter 9,
“Disaster Recovery” on page 453.

64
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.3.3 Creating a consistent point-in-time copy
While the copy pair volumes are in the duplex pending XD state, the secondary volumes
maintain a
fuzzy copy
of the data:
Because of the non-synchronous data transfer characteristics, at any time there will be a
certain amount of updated data that is not reflected at the secondary volume. This data
corresponds to the sectors that were updated since the last volume bitmap scan was
done. These are the out-of-sync sectors.
Because of the bitmap scan method, writes are not ensured to be applied on to the
secondary volume in the same sequence as they are written to the primary volume.
When terminating the PPRC relationship to establish host access to secondary volumes, the
first issue may cause loss of transactions. Since a file system’s or database’s consistency
depends on the correct ordering of write sequences, the second issue may cause
inconsistent volumes. Therefore, to use secondary volumes by the host systems, you need to
make them
point-in-time consistent
:
The secondary volumes have to
catch up
to their primary counterparts. PPRC-XD
catch-up is the name of the transition that occurs to a PPRC-XD pair when it goes from its
normal out-of-sync condition until it reaches a full sync condition. At the end of this
transition, the primary and secondary volumes become fully synchronized.
Then the application needs to be quiesced and the PPRC volume pairs temporarily
suspended. This is necessary to ensure consistency not only at the volume level, but also
at the application level.
A FlashCopy of the secondary volumes onto tertiary volumes should now be performed,
followed by resuming of the PPRC-XD pairs.
These tertiary volumes are then a consistent point-in-time copy of the primary volumes.
Figure 4-6 on page 65 summarizes this procedure, which provides a consistent point-in-time
copy of the data:
1.Quiesce the application updates.
2.Synchronize the volume pairs by one of these methods:
– Perform the catch-up by issuing a
go-to-sync operation
. This is done by the PPRC task
Establish Synchronous PPRC copy pair with the option Copy out-of-sync
cylinders only to minimize the time needed for data transfer.
Now the volume pair leaves the duplex pending XD state and will reach the duplex
state (refer back to Figure 4-5 on page 62). From this moment, if the pairs were not
immediately suspended, primary write updates would be synchronously transmitted to
the recovery site.
You may not want any synchronous copy operations to occur once the volumes are in
duplex state, especially if the volumes you are mirroring are separated by long
distances (beyond 103 km with ESCON or beyond 300 km with PPRC over Fibre
Channel). For this reason, you may select the additional option Suspend PPRC after
establish complete when creating the Establish Synchronous PPRC copy pair
task.
– Perform the catch-up by waiting until all application updates are transferred to the
secondary site.
You can monitor the number of out-of-sync tracks in the volume information panel or by
issuing rsQuery commands (see Chapter 7, “ESS Command Line Interface” on page 377).
If you triggered the catch-up by issuing
go-to-sync
, then you can also watch for completion
of the Synchronous PPRC task using the rsQueryComplete command.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
65
3.Suspend the PPRC copy pairs after they reach the full duplex state. (With a
go-to-sync

operation, you can select the additional task option Suspend PPRC after establish
complete to avoid this step.) If you use PPRC Consistency Groups (this will be discussed
in 4.6, “PPRC data consistency” on page 84), issue a Freeze operation.
At this point, we have a set of consistent secondary volumes.
4.You can resume the application (updates will not be transferred to the secondary volumes
because the pairs are suspended).
5.Perform a FlashCopy on the secondary volumes.
6.Resume PPRC Extended Distance mode for the copy pair.
Figure 4-6 Create a PPRC-XD consistent copy
For application recovery based on point-in-time copies, you have to plan for appropriate
checkpoints to briefly quiesce the application and synchronize the volumes pairs. When the
recovery of the application is done, you must remember that, while in an active PPRC-XD
relationship, the secondary volumes always have a current fuzzy copy of the primary
volumes, so you have to keep the tertiary volumes where you
FlashCopied
the last globally
consistent catch-up. As you can realize, this tertiary copy will not be reflective of the current
update, but rather, any updates up until the last global catch-up operation.
4.3.4 Synchronous PPRC and PPRC-XD comparison
Synchronous PPRC and PPRC-XD differ by several characteristics. Some of the more
important ones are data consistency and currency of data at the secondary site, impact of
channel
extender
channel
extender
F
l
a
s
h
C
o
p
y
secondary
tertiary
primary
non-synchronous
PPRC
over long distance
5.
FlashCopy
.
6.
Reestablish
suspended
pairs (resync).
consistent tertiary
copy of data
Secondary Site
Primary Site
minimum performance impact fuzzy copy of data
1.
Quiesce
application
updates.
2.
Catch-up
(synchronize
volume pairs).
go-to-sync / suspend
or
wait
for application
writes to quiesce
3.
Build consistency
on
recovery data.
resync and freeze
or
freeze
(suspend)
4.
Resume
application writes
as soon as freeze is done.
individual volume pairs synchronize
copy of data consistent across
volume pairs

66
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
latency for applications, and bandwidth requirements. Table 4-2 summarizes these
differences.
Table 4-2 Comparison of Synchronous PPRC and PPRC-XD characteristics
Synchronous PPRC provides a consistent copy, which means that the secondary volume is
as good as possible a complete match of the primary volume. Dependent writes are applied
on the secondary volumes in the same sequence as they are applied on the primary volumes.
This makes Synchronous PPRC ideal for rolling disaster recovery scenarios. PPRC-XD is an
asynchronous data mover that provides a fuzzy copy of the data on the secondary volume. As
a consequence, periodic go-to-sync operations to build consistent point-in-time copies at the
recovery site are necessary. Due to this, recovery is possible to the last point-in-time copy.
Because PPRC-XD does not delay the acknowledgement of a write complete until updates
have been made to the secondary volume, PPRC-XD preserves the application write
performance better than Synchronous PPRC over longer distances. With PPRC Extended
Distance, the ESSs hosting the primary and secondary volumes can be separated by
distances well beyond the 103 km (300 km with PPRC over Fibre Channel) supported with
PPRC synchronous transmissions (continental distances) and are only limited by the channel
extender and network technology capabilities. As synchronous mirroring cannot occur over
long distances without a serious application performance impact, the best trade-off is to
accept the periodical brief application quiesces, in order to build consistent point-in-time
copies of the data. Another way may be to combine Synchronous PPRC and PPRC-XD in a
cascading relationship (see 4.4, “Asynchronous Cascading PPRC” on page 67).
Application write performance under Synchronous PPRC is dependent on the available
bandwidth, so only critical data should be copied. PPRC-XD allows you to better exploit your
available bandwidth capacity, therefore allowing you to include more of your data to be
protected.
As summarized in Figure 4-7 on page 67, PPRC Extended Distance is a recommended
solution for data copy, data migration, off-site backup, and transmission of inactive database
logs with excellent performance, which is particularly relevant when implemented over
continental distances. PPRC Extended Distance can also be used for application recovery
solutions based on periodic point-in-time backup copies of the data if the application tolerates
periodic brief interruptions (application quiesce).
Synchronous PPRC PPRC Extended Distance
If write W
1
completes at primary site before W
2
is
initiated, then secondary site never has
modifications from W
2
without also having W
1
.
No consistency guarantee; point-in-time
consistency with quiesce/resume procedure.
Data at the secondary site is up-to-date when
write has been acknowledged to host.
Data transfer delayed (indefinite amount of time)
after write has been acknowledged to host.
Latency impact for applications increases
proportionally with distance between sites.
Latency impact is independent of the distance
between sites.
Bandwidth must be sufficient to carry the
instantaneous peak write load.
Bandwidth must be greater than the average
write bandwidth.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
67
Figure 4-7 PPRC Extended Distance positioning
For an in-depth discussion of PPRC characteristics, see “Advanced functions for storage
subsystems: Supporting continuous availability” by Azagury, et al, in IBM Systems Journal,
Vol.42 (2003) No.2 (June), found at:
http://www.research.ibm.com/journal/sj/422/azagury.pdf
4.4 Asynchronous Cascading PPRC
A feature introduced with ESS Copy Services V2 called Asynchronous Cascading PPRC
allows you to combine Synchronous PPRC and PPRC-XD, providing another method for
Disaster Recovery.
4.4.1 Asynchronous Cascading PPRC operation and volume states
With Asynchronous Cascading PPRC, a volume is allowed to be simultaneously both a PPRC
primary and a PPRC secondary, combining three volumes in two PPRC pairs. Figure 4-8 on
page 68 shows the principles of this configuration.
PPRC Extended Distance Positioning
It can be used over continental distances with excellent application performance.
The distances are only limited by the network and channel extenders'
capabilities.
Applications' write operations do not have distance-dependent latency impact.
Fuzzy copy of data at the recovery site (sequence of dependent writes may not be
respected at the recovery site).
Recovery data can become a consistent point in time mirror of the primary data, if
appropiate application checkpoints are set to do global catch-ups.
Pairs are synchronized with application group consistency.
Synchronizations can be done more frequently, because of short catch-ups
RPO still non-zero, but improves substantially.
PPRC Extended Distance is a recommended solution for data copy, data
migration and offsite backup — over continental distances — with excellent
application performance.
It can be used for disaster recovery implementations if the application can be
quiesced and non-zero data loss Recovery Point Objective (RPO) is
admissible.

68
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 4-8 Volume pairs in an Asynchronous Cascading PPRC relationship
In Figure 4-8, we have two PPRC volume pairs: A–B and B–C. Volume B acts in two roles
simultaneously: It is the secondary volume of the PPRC relationship A–B and the primary
volume of the PPRC relationship B–C. As with any PPRC relationship, volumes A and B, as
well as volumes B and C, must be in a different LSS (in the same or different ESS). There is a
minimum of two different ESSs and a maximum of three different ESSs in an Asynchronous
Cascading PPRC relationship. Figure 4-9 shows the volume panel of an ESS with LSS 16 as
local and LSS 17 as the intermediate LSS. Volume 700-18767 has two status icons: the red
as secondary of the local Synchronous PPRC relationship, the blue as primary of the remote
PPRC-XD relationship.
Figure 4-9 Asynchronous Cascading PPRC intermediate volume states
In practice, the three volumes will be often in different sites, called the
local
, the
intermediate
,
and the
remote site
. Accordingly, we denote the pair A–B as the
local pair
and B–C as the
Remote Pair
Volume B
Secondary
Primary
Secondary
Primary
Local Pair
Intermediate
ESS
Local
ESS
Remote
ESS
Host I/O
Volume CVolume A

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
69
remote pair
. When considering the whole chain of data transfer, volume A can be considered
as the primary, volume B as the secondary, and volume C as a
tertiary
volume.
The state of both PPRC pairs is maintained independently. You can suspend or resume A–B
without changing the state of B–C and vice versa. It is important to note that volume B has
two PPRC states, depending on which relationship is considered. Figure 4-10 shows the
volume information panel of the intermediate volume from the example above, indicating that
this volume acts as a PPRC primary as well as a secondary.
Figure 4-10 Asynchronous Cascading PPRC intermediate volume information
4.4.2 Asynchronous Cascading PPRC tasks and options
To maintain Asynchronous Cascading PPRC relationships, you apply the same types of ESS
Copy Services tasks to establish, suspend, and terminate PPRC for each of the two pairs
separately. The available options depend on the mode of operation you use for the specific
volume pair (see 4.2.4, “Synchronous PPRC tasks and options” on page 60 and 4.3.2,
“PPRC-XD tasks and options” on page 62). The only difference is that you must select the
option Asynchronous Cascading PPRC when establishing the B–C pair:
Asynchronous Cascading PPRC allows the PPRC primary volume of that relationship to be
also the secondary volume in another PPRC relationship.
The cascading pairs may be established in any order. We recommended the following
procedure:
1.Establish the remote pair with the option Do not copy volume.
Note: There is no additional volume involved in the intermediate LSS. With respect to the
total costs of ownership, this saves disk space and license fees compared with solutions
based on an additional FlashCopy volume in the intermediate site.

70
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.Establish the local pair with the option Copy entire volume.
This avoids the unnecessary transfer of invalid data from the secondary to the tertiary
volume. If you alternatively establish the local pair first with Copy entire volume option, then
you have to specify Copy entire volume also for the remote pair. A cascading relationship
may be added to existing pairs too.
4.4.3 Asynchronous Cascading PPRC mode combinations
Table 4-3 shows the supported combinations of PPRC operation modes in a cascading
relationship.
Table 4-3 Asynchronous Cascading PPRC mode combinations
The combination PPRC Extended Distance to Synchronous PPRC is not supported because
the intermediate volume as a fuzzy copy cannot be the source for a consistent copy at the
tertiary volume.
Synchronous PPRC to PPRC-XD is the typical usage intended for Disaster Recovery. The
combination PPRC-XD to PPRC-XD allows you to maintain multiple copies of data without
significant impact to application I/O performance. In contrast to that, a combination of two
Synchronous PPRC relationships will have a severe impact to application I/O because each
write operation has to wait for two PPRC transfers. This combination may be useful to
guarantee a consistent copy at the remote site if there is little write I/O and the whole distance
is beyond the maximum distance of 103 km (300 km with PPRC over Fibre Channel)
supported for a single Synchronous PPRC.
Although our terminology for local, intermediate, and remote ESSs may suggest that
Asynchronous Cascading PPRC requires three ESSs, there are other variants possible. The
only one requirement is that the two volumes forming a basic relationship A–B or B–C must
be in different Logical Subsystems. These LSSs may be in the same or in different ESSs, so
there are variants of Asynchronous Cascading PPRC with three LSSs in the same, in one or
two ESS(s). From a practical point of view, only two variants make sense:
Three ESSs with one volume per ESS, as shown in Figure 4-8 on page 68.
When the local pair runs in synchronous mode, you should keep the distance between the
local and intermediate ESSs as close as possible to minimize the performance impact.
The intermediate ESS should be in a secure environment separated from the local ESS
and with separate power to reduce the possibility of an outage affecting both locations.
The remote ESS is typically in a different city, any distance away from the intermediate
ESS. If the remote PPRC pair operates asynchronously from the local PPRC pair, then
the effect of long distance on host response time is minimized.
Two ESSs with volumes A and B in the same local ESS and volume C in the remote ESS,
as shown in Figure 4-11 on page 71.
If the remote pair operates in PPRC-XD mode, this configuration is very similar to a single
PPRC-XD relationship. Having the intermediate volume leads to procedures for creating a
consistent remote point-in-time copy with less application impact.
Local volume pair Remote volume pair
Synchronous PPRC PPRC Extended Distance
Synchronous PPRC
PPRC Extended Distance PPRC Extended Distance

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
71
Figure 4-11 Asynchronous Cascading PPRC with two sites
4.4.4 Creating a consistent copy with PPRC-XD as a remote pair
The combination of Synchronous PPRC and PPRC-XD brings up again the question of how
to create a consistent copy at the remote site. As discussed in 4.3.3, “Creating a consistent
point-in-time copy” on page 64, the secondary volume of a PPRC-XD relationship is not
consistent and generally not useful for recovery unless all of the data has been copied up to
some point in time. PPRC-XD secondary volumes must periodically be made consistent and
an additional copy of the data kept.
We describe how to create a consistent backup copy of volume A at the remote site under the
following assumptions:
The data on volumes A and B is identical (volume state is full-duplex).
Volumes A and B were established in PPRC synchronous mode.
Volumes B and C were established in PPRC Extended Distance mode.
Under these assumptions, the following procedure (summarized in Figure 4-12 on page 72)
creates a consistent point-in-time copy at the remote site:
1.At the local ESS:
Issue a Freeze operation (see “Freeze operation” on page 90) to the A–B volume pair.
This ensures that no further updates occur to volume B and updates are allowed to
complete on volume C. Volumes A and B (the Synchronous PPRC pair) are now
suspended in a consistent state.
2.At the intermediate ESS (in a two-ESS configuration, the local ESS):
a.Convert volumes B and C to synchronous mode to allow pending tracks and updates
to be transmitted to the remote site.
b.Monitor the copy process of volumes B and C for the transition to full duplex state.
When volumes B and C are in full duplex state, you can create a copy at the remote ESS
that is consistent to the time of the freeze.
Remote Pair
Volume B
Secondary
Primary
Secondary
Primary
Local Pair
Local
ESS
Remote
ESS
Host I/O
Volume CVolume A

72
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
3.At the remote ESS:
While volumes A and B are suspended and after the synchronization process of
volumes B and C is complete, use FlashCopy to create a point-in-time consistent copy
using volumes C and D.
4.At the intermediate ESS (in a two-site configuration, the local ESS):
a.Suspend volumes B and C.
b.With volumes B and C in a suspended state, reestablish volumes B and C as a
PPRC-XD pair. Select the Copy out-of-sync cylinders only option.
5.At the local ESS:
a.Resume operations after the freeze. When you issue the freeze to volume A, the
established paths between the LSSs were disabled. You have to reestablish the paths.
b.Resynchronize the suspended A–B volume pair so that all the changed data is copied
from volume A to B. Select the Copy out-of-sync cylinders only option.
Figure 4-12 Create consistent copy with Asynchronous Cascading PPRC
Note: When using Fibre Channel links between site A and site B, if the Synchronous
PPRC is active with application I/O still running at site A, and if a go-to-SYNC is
triggered for the XD pair, then the XD pair will automatically suspend.
We recommend that when go-to-SYNC is issued for the XD pair, the Synchronous
PPRC pairs should be suspended (this will normally be done via a freeze operation, as
described in our example).
Primary site
Intermediate site
Remote site
FlashCopy
Primary
Primary
Secondary
Secondary
Synchronous PPRC
PPRC-XD
FlashCopy
5.With synchronous pair:
a.
Resume
after freeze
(reestablish paths)
b.
Resynchronize
with
Copy out-of-sync
1.With synchronous pair:
Freeze
2.With PPRC-XD pair:
a.
Convert
to
synchronous
b.
Wait
for full duplex
consistent
suspended
consistent
3.
FlashCopy
4.With PPRC-XD pair:
a.
Suspend
b.
Reestablish
as
PPRC-XD

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
73
Reviewing this procedure, you should notice the following details:

Synchronous PPRC Consistency Groups need to be implemented in the standard way for
the Synchronous PPRC relation (step 1 on page 71 of the procedure).

Once the PPRC Freeze operation happens, the intermediate site volumes become static
and consistent; therefore, as soon as the bandwidth permits, the remote site will catch up
also to this static and consistent point. So step 2 on page 71 is not really an action, it is
only a process to watch for.

The FlashCopy at the remote ESS (step 3 on page 72) can be issued as Inband
FlashCopy.
4.4.5 Outage scenarios with Asynchronous Cascading PPRC
The outage scenarios with Asynchronous Cascading PPRC depend on the configuration with
two or three ESSs:

Asynchronous Cascading PPRC with local and intermediate volumes in the same ESS
leads to the same scenarios as a single PPRC configuration. After losing access to the
local ESS, the remote ESS provides a fuzzy copy if the remote pair was in a PPRC-XD
relationship. (Operating both PPRC relations in synchronous mode does not make sense
in a two-ESS configuration.)
Recovery has to be made with the last consistent point-in-time copy in the remote ESS.
Therefore, when we compare configurations of two ESSs running Asynchronous
Cascading PPRC or a single PPRC, we do not see increased availability or recovery. But
there is less impact to the applications with Asynchronous Cascading PPRC when
creating a consistent point-in-time copy at the remote site (compared with single
PPRC-XD).

Asynchronous Cascading PPRC with three ESSs offers more options:
– If there is an outage of the local ESS, the intermediate ESS contains a consistent and
up-to-date copy of the data. You will fail over production to the intermediate ESS
without loss of transactions.
Shortly (depending on bandwidth), the intermediate ESS will complete sending all
updates to the remote ESS. In this way, the remote site will automatically catch up.
– If there is an outage of the intermediate ESS, data at the local ESS is not affected.
Applications continue to run as normal.
There is only one copy of data available until the intermediate ESS has been repaired.
– If there is an outage at the remote ESS, data at the local and intermediate ESS is not
affected. Applications continue to run as normal. The intermediate ESS maintains a
consistent up-to-date copy.
You may ask the question, what happens if you lose both the local and the intermediate ESS
of a three ESS Asynchronous Cascading PPRC configuration? The answer is that you go
back to the most recent point-in-time copy that was made at the remote ESS. In other words,
the situation is the same as losing the primary ESS in a two-ESS PPRC-XD configuration. If
the outage is a disaster large enough to cause the loss of both the primary and the
intermediate ESS, then there are probably problems affecting the whole region and the
business. In such situations, the larger time needed to recover the data will not be the limiting
factor for business contingency.

74
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.4.6 PPRC Failover and Failback with Asynchronous Cascading PPRC
Asynchronous Cascading PPRC has been enhanced to exploit PPRC Failover and Failback
modes in a three-site solution. This can help reduce the time required to synchronize PPRC
volumes after switching between sites during planned or unplanned outages.
This PPRC enhancement is supported on ESS Models 800, 750, F20, and F10, with LIC level
2.3.0 or above, and is provided with PPRC Version 2 (PPRC Version 2 is an optional feature
to the ESS).
PPRC Failover and Failback with Asynchronous Cascading PPRC is discussed in detail in
9.2, “Data consistency” on page 458.
4.5 Asynchronous PPRC
Asynchronous PPRC is designed to provide a long-distance remote copy solution across two
sites for open systems and z/OS data using asynchronous technology. It is based on existing
PPRC-XD and FlashCopy Copy Services functions. It has an additional functionality for a
periodical creation of Consistency Groups across one or more ESSs and for the management
and control of the Asynchronous PPRC session.
It operates over high-speed Fibre Channel communication links and supports virtually an
unlimited distance between the local and remote sites. The distance is typically limited only
by the capabilities of the network and channel extension technologies. This can better enable
you to choose your remote site location based on business needs and enables site
separation to add protection from localized disasters.
4.5.1 Asynchronous PPRC overview
In the Asynchronous PPRC environment, the data at the remote site is maintained to be a
point-in-time consistent copy of the data at the local site, which is done without significant
impact to applications at the local site. Compared to PPRC Extended Distance,
Asynchronous PPRC provides automation to periodically create Consistency Groups at the
primary volumes and to FlashCopy the consistent data at the secondary site to the tertiary
volumes.
Data currency at the remote site lags behind the local site an average of 3 to 5 seconds,
minimizing the amount of data exposure in the event of an unplanned outage. Data currency
is also known as the recovery point objective (RPO). The actual lag in data currency
experienced will depend upon a number of factors, including specific workload characteristics
and available bandwidth between the local and remote sites.
Data consistency at the remote site is internally managed across up to a total of eight ESSs
distributed across the local, primary and remote, secondary sites.
The required RPO can be dynamically changed as a parameter of the Asynchronous PPRC
Session according to the requirements based on the RPO and the bandwidth (for more
information about RPO, see Chapter 9, “Disaster Recovery” on page 453.)
4.5.2 Asynchronous PPRC terminology
In this section, we discuss the new terms used when working in an Asynchronous PPRC
environment.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
75
Asynchronous PPRC session
An Asynchronous PPRC Session is a collection of volumes that are managed together when
creating consistent copies of data. This set of volumes can reside in one or more LSSs and
one or more ESSs in the domain. The volumes are located at the customer’s primary site.
When you start or resume a session, the creation of Consistency Groups is started and the
Master ESS manages the session by communicating with the Subordinate ESSs.
Master
The Master is a term for a function inside an ESS that communicates with the Subordinate
ESSs to control the creation of Consistency Groups and manage the Asynchronous PPRC
Session. The Master is defined when the Start/Resume Asynchronous PPRC Session
command is issued to an LSS in the Master ESS.
The Master ESS needs PPRC Fibre Channel communication paths to any of the LSSs in
each Subordinate ESS that has volumes as part of the session.
Subordinate
The Subordinate is a term for a function inside an ESS that communicates with the Master
and is controlled by the Master. One of the LSSs of each Subordinate ESS needs to have
PPRC Fibre Channel communication paths from the Master ESS established. It is used to
enable the Master to create Consistency Groups using volumes in different ESS boxes.
If all the volumes of an Asynchronous PPRC Session reside in one primary ESS, no
Subordinate is required because the Master ESS can communicate to all LSSs inside the
primary ESS.
4.5.3 Asynchronous PPRC operation and volume states
The purpose of Asynchronous PPRC is to provide a consistent copy of production data at the
remote site with good currency and no significant performance impact. Some of the reasons
for using Asynchronous PPRC would be:

The distance between primary and secondary sites is beyond 300 km (which is the
longest supported distance with Synchronous PPRC over Fibre Channel links).

The distance between primary and secondary sites is less then 300 km, but the
performance impact of synchronous processing is not admissible.
The Asynchronous PPRC uses some of the old ESS Copy Services functions and also adds
some new functionality in terms of managing the Asynchronous PPRC Session. PPRC
Extended Distance is used for draining the data from the primary volumes to the secondary
volumes at the remote site (PPRC Extended Distance is discussed in 4.3, “PPRC Extended
Distance (PPRC-XD)” on page 61). FlashCopy is used to copy the point-in-time consistent
data from secondary to the tertiary volumes (FlashCopy function is discussed in chapter
Chapter 3, “FlashCopy” on page 35).
PPRC-XD
For better understanding, before we describe how Asynchronous PPRC works, let us review
the steps needed for creation of a consistent point-in-time copy of data at the remote site
when using only PPRC-XD (for detailed information, refer to 4.3.3, “Creating a consistent
point-in-time copy” on page 64). Figure 4-13 on page 76 shows the PPRC-XD environment
while the steps are provided below.

76
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 4-13 PPRC-XD environment

Quiesce of the application updates

Synchronization of the volume pairs, either by using the
go-to-sync
procedure or by
waiting for all the data to propagate to the secondary volumes

Suspension of the volume pairs when the volumes are in duplex state or synchronized

Resumption of the application updates

FlashCopy of secondary volumes to the tertiary volumes

Resumption of the PPRC-XD pairs with the Copy out-of-sync cylinders only option
These steps can be done manually or can be automated using scripts. The data currency
depends on how often the consistent point-in-time copy of data is created at the secondary
site. As there are many steps to be performed for creation of consistent data, it is not a
recommended solution for environments where the consistent data is created at very short
periods of time (the data currency at the remote site is very important).
How Asynchronous PPRC works
The new process of Asynchronous PPRC is performed by the Licensed Internal Code 2.4.x or
higher of the ESS and works as described in Figure 4-14 on page 77 for all volumes in one or
more LSSs that have been previously added to a defined Session.
channel
extender
channel
extender
Fl
a
s
h
C
op
y
secondary
tertiary
primary
non-synchronous
PPRC
over long distance
consistent tertiary
copy of data
Secondary Site
Primary Site
minimum performance impact
fuzzy copy of data

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
77
Figure 4-14 Asynchronous PPRC processing
The A volumes at the local site are the production volumes and are used as PPRC-XD
primary volumes. The data from the A volumes is replicated to the B volumes, which are
PPRC-XD secondary volumes. At a certain point in time, a Consistency Group is created
using all of the A volumes, even if they are located in different ESS boxes. This has no
application impact as the creation of the Consistency Group is very quick (in the order of
milliseconds). Once the Consistency Group is created, the application writes can continue
updating the A volumes. The increment of the consistent data is sent to the B volumes using
the existing PPRC-XD relationship. Once the data reaches the B volumes, it is FlashCopied
to the C volumes. The C volumes now contain the
application consistent
copy of data.
Because the B volumes usually contain a
fuzzy
copy of the data from the local site (not when
doing the FlashCopy), the C volumes are used to hold the last point-in-time consistent data
while the B volumes are being updated by the PPRC-XD relationship.
The data at the remote site is current within three to five seconds, but this depends on the
workload and bandwidth available to the remote site.
Note that the consistent data at the remote side resides on the FlashCopy target volumes (the
C volumes) and not on the PPRC Target volume.
In contrast to the previously mentioned PPRC-XD solution, the Asynchronous PPRC
overcomes its disadvantages and automates all of the steps that have to be done manually
when using PPRC-XD. It provides:

A very good data currency: The data at the remote site is usually only a few seconds
behind the data at the local site.

Automatic creation of consistent data: The Consistency Groups are created automatically
in short periods of time. This can be changed with a session parameter, which will be
discussed later in this chapter.
1. Create Consistency Group of volumes at local site
2. Send increment of consistent data to remote site
3. FlashCopy at the remote site
4. Resume PPRC-XD (copy out-of-sync data only)
5. Repeat all the steps according to the defined time period
Asynchronous PPRC - How it works

PPRC-XD
Local Site Remote Site
PPRC Primary PPRC Secondary
FlashCopy Source
C
B
A
FlashCopy
Target
FlashCopy
Automatic Cycle in an active Asynchronous PPRC Session

78
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

No application impact: There is no application impact as the Consistency Group formation
is very fast (in the order of milliseconds).

Single point of control: The Asynchronous PPRC Session is managed from the Master
ESS, even when the production data is spread across more than one ESS.
To enable a very fast creation of Consistency Groups across multiple ESSs, a fast inband
communication network is required. The following configuration example, Figure 4-15, shows
the necessary communication across LSSs and ESSs.
Figure 4-15 Asynchronous PPRC Configuration
One of the local site ESSs is the Master ESS that communicates to Subordinate ESSs over
Fibre Channel links to enable the creation and control of the Consistency Groups. The Fibre
Channel communication paths have to be defined from one or more LSSs inside the Master
ESS to at least one LSS in each Subordinate ESS. The paths can be established with the
same commands as PPRC data paths. They can be defined on separate physical links or on
links combined with PPRC data paths usually in a SAN. For redundancy purposes at least
two paths from the Master to each Subordinate should be defined.
The volumes of the Master ESS do not have to be members of an Asynchronous PPRC
session, but they may be.
The volumes in the session can be open systems volumes as well as z/OS volumes.
Consistency Groups are created for all volumes at the same time.
Asynchronous PPRC volume states
There are three new volume states and corresponding icons related to the Asynchronous
PPRC Session. For more information about the volume states and icons, refer to 6.5.1,
“Volumes panel icons” on page 201.
Asynchronous PPRC Configuration Example
Remote Site
Master
Subordinate
Subordinate
LSS
LSS
LSS
LSS
Local Site
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
PPRC-XD
Relations
FC
communication
Links
SAN
Fabric
LSS
LSS
LSS
LSS
Subordinate
PPRC Secondary
Volumes
FlashCopy Target
Volumes
C
B
A
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
79
The volume states are following:

Volume Session Active: This volume state shows that the volume has already joined the
Asynchronous PPRC Session and is used for the creation of Consistency Groups.

Volume Session Join Pending: This volume state shows that the volume has been
added to the Asynchronous PPRC Session but the first copy of data between the
PPRC-XD pairs has not finished yet. The volume is not yet used for creation of the
Consistency Groups.

Volume Session Remove Pending: This volume state shows that the volume is being
removed from the Asynchronous PPRC Session.
4.5.4 Asynchronous PPRC tasks and options
In this section, we describe the Copy Services tasks and their options used for managing the
Asynchronous PPRC environment. For detailed information on how to use these tasks, refer
to Chapter 6, “ESS Copy Services Web User Interface: LIC 2.2.0 and later” on page 189.
Manage session
A session must be opened for each Logical Subsystem (LSS) that participates in the
Asynchronous PPRC environment. If a session is opened on an LSS, it means that you can
add volumes from that LSS to the Asynchronous PPRC Session. When opening the session,
you also have to provide a unique Session ID. Close the session on the LSS that has no
volumes in the session and it will no longer participate in the Asynchronous PPRC
environment.
For the Open Session and Close Session tasks shown below, you must also provide the
Session ID using one of the following options:

Select Session Id: Use this option to select an existing session ID from the list. At least
one session must be defined in the Copy Services Domain.

Enter Session Id [1-255]: Use this option to enter the session ID manually.
Open Session: This task opens an Asynchronous PPRC Session on an LSS.
Close Session: This task closes the session with the LSS you have selected.
Manage Session Members
To add volumes to the session you have to specify all the PPRC primary volumes that you
intend to use in the Asynchronous PPRC environment. Each volume will be in
join pending

state until their first PPRC synchronization pass has finished. The volumes can be
dynamically removed and added from or to a running session. For more information about
adding or removing volumes to and from the session, see “Add volumes to session” on
page 342.
To manage Asynchronous PPRC Session members, you have to use one of the following
tasks: Add Volume to Session or Remove Volume from Session. For these tasks, you
must also provide the Session ID using one of the following options:

Select Session Id: Use this option to select an existing session ID from the list. At least
one session must be defined in the Copy Services Domain.
Note: The Manage Session Members task can be issued at the LSS level or at the
volume level. If issued at the LSS level, it will add all volumes from the LSS to the session.
If the task is issued on the volume level, it will add just the selected volumes to the session.

80
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

Enter Session Id [1-255]: Use this option to enter the session ID manually. You must
provide an existing session ID here for the command to work.
Add Volume to Session: This task adds selected volumes or all volumes in the selected
LSS to the Asynchronous PPRC Session.
Remove Volume from Session: This task removes selected volumes or all volumes in the
selected LSS from the Asynchronous PPRC Session.
Start/Resume Async PPRC
When you issue this command, you have to select an LSS in one of the ESSs. The ESS
containing the LSS to which you issue the Start session command becomes the Master. If
you want to resume a previously paused session, you have to issue the Resume command to
the same LSS on which the session was started. If you are planning to use Subordinate
ESSs, then when issuing the Start session command you also have to provide
Master/Subordinate path associations. For more information on how to start an
Asynchronous PPRC Session, refer to 6.17.4, “Starting Asynchronous PPRC” on page 344.
There are three parameters that can be set when starting or resuming the Asynchronous
PPRC Session:

Consistency Group Timer (in seconds, default 0, maximum 64K sec), determines the
time from last successful or failed FlashCopy to the start of the creation of the next
Consistency Group.

Maximum Coordination Timer (in milliseconds, default 50 ms, maximum 64,000 ms),
determines the maximum time for the creation of a Consistency Group at the local site.

Maximum Consistency Group Drain Time (in seconds, default 30 sec, maximum 64,
000 sec), determines the maximum time to transfer the consistent data to the remote site.
These parameters may be changed when resuming the Asynchronous PPRC Session by
using the Modify timer values (for Resume only) check box.
Pause/Terminate Async PPRC
This command can be used to terminate or pause the Asynchronous PPRC Session. Both
commands stop the Asynchronous PPRC Session.
Pause will ensure the current Consistency Group forms before stopping the Asynchronous
PPRC Session.
Terminate will interrupt the formation of a Consistency Group if one is running and clean up
by reverting to the previous Consistency Group. If no group is being formed, it will terminate
immediately. Use Terminate when you need to change the session topology, such as defining
a new Master and adding or removing a Subordinate. The command removes the
Master/Subordinate relationship information.
Force Terminate will terminate immediately and will interrupt the creation of the Consistency
Group without recovery if one is being processed.
The following Task Types can be selected:

Pause: This task type will pause the Asynchronous PPRC Session.

Terminate: This task type will terminate the Asynchronous PPRC Session.

Force Terminate: This task will immediately terminate the Asynchronous PPRC Session
without recovery.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
81
Topology options:

Master: Select this option if you are issuing the command to the Master ESS.

Subordinate: Select this option if you are issuing the command to the Subordinate ESS.
You must also provide the session ID using one of the following options:

Select Session Id: Use this option to select an existing session ID from the list. At least
one session must be defined in the Copy Services Domain.

Enter Session Id [1-255]: Use this option to enter the Session ID manually. You must
provide the existing session ID here for the command to work.
4.5.5 Setting up Asynchronous PPRC
In this section, we describe the steps needed to set up the Asynchronous PPRC
environment. We also include a list of prerequisites.
Prerequisites for the Asynchronous PPRC

ESS model 800 or 750 for the primary and secondary subsystems.

PPRC V2 for primary and secondary ESSs.

Fibre PPRC links.

ESS LIC 2.4.x or higher.

New ESS machines must have the Feature code #9912 Asynchronous PPRC Indicator.
– Existing ESS machines do not require the new feature code.

FlashCopy V2 required at the secondary ESS.

If you intend to run Asynchronous PPRC from the secondary site to the primary site (for
example, in case of a disaster), then FlashCopy V2 is also required on the primary ESS.

Any site with PPRC V2 that uses FlashCopy requires FlashCopy V2 (see Appendix B,
“Copy Services feature codes” on page 541).
Asynchronous PPRC setup steps
First of all, identify all PPRC primary and secondary volumes and the corresponding LSSs as
well as the FlashCopy target volumes at the remote site for the Asynchronous PPRC session.
Determine the Master ESS and LSS and the Session ID. See Figure 4-16 on page 82 for the
Initialization steps.
1.Establish PPRC data paths from primary ESSs to secondary ESSs.
2.Establish PPRC-XD volume pairs from primary to secondary ESSs.
3.Establish FlashCopy relationships from B volumes to the C volumes at the remote site.
Use the following options: No Background copy, Start Change Recording, Inhibit writes to
target, and Persistent FlashCopy.
4.Establish control paths from the Master ESS to each of the Subordinate ESSs.
5.Open a session with all LSSs that will participate in the Asynchronous PPRC environment
(only the PPRC source volumes).
6.Add all PPRC primary volumes to the session.
Note: Force should only be done when instructed to do so by IBM support. It requires
that the Consistency Group be manually verified before the data can be used.

82
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
7.Start the Asynchronous PPRC session using one of the LSSs located in the Master ESS.
Figure 4-16 Asynchronous PPRC setup steps
4.5.6 Asynchronous PPRC management
A session is a collection of volumes in an ESS or in multiple ESSs that are managed together
when creating consistent copies of data. When you start or resume a session, the
Asynchronous PPRC configuration is specified or modified. This causes Asynchronous
PPRC to start forming the Consistency Groups for each volume in the session. The topology
of a session is defined by its Master/Subordinate relationship.
At the time of General Availability of LIC V2.4.0 support, with open systems, only the Copy
Services Web User Interface and the Command Line Interface can be used to manage
Asynchronous PPRC. In this redbook, we focus on the management using the Web User
Interface, but we also explain the capabilities of the Command Line Interface, which can be
easily used for automation purposes.
The IBM Multiple Device Manager Replication Manager has been previewed to be able to be
used for managing Asynchronous PPRC in the future (see 9.9, “Multiple Device Manager
Replication Manager” on page 505 for more information about IBM Multiple Device Manager;
also refer to the IBM TotalStorage Multiple Device Manager Usage Guide, SG24-7097).
Open systems and z/OS volumes are managed in the same session and the creation of the
Consistency Groups is one process for both platforms. With the Web User Interface, you can
manage both types of volumes.
It is also possible to establish an Asynchronous PPRC relationship in both directions with
primary and secondary volumes at either the local or remote sites.
Asynchronous PPRC Initialization
Remote Site
Master

Subordinate
Subordinate
LSS
LSS
LSS
LSS
Local Site
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
FC
communication
Links
SAN
Fabric
LSS
LSS
LSS
LSS

Subordinate
C
B
A
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
LSS
1. Establish
PPRC Paths
5.Create Session
for each LSS
3.Establish
FlashCopy Pairs
2.Establish
PPRC-XD Pairs
4. Establish control
paths between
Master to Subordinate
7.Start Asynchronous
PPRC with command
to the Master
6. Add Volumes
to the session

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
83
Restrictions

Currently, only one Master can reside in an ESS and therefore only one Session can be
managed by one ESS. (Architecture allows for growth.)

An LSS can be assigned only to one Session. (Architecture allows for growth.)

Maximum of eight ESSs (in any combination of primaries and secondaries) can participate
in an Asynchronous PPRC environment using Web User Interface.
The Asynchronous PPRC mechanism will only act upon volumes that have been added to a
Session. Other PPRC volumes may still participate in any other Copy Services relationship
and not be included in the formation of Consistency Groups.
4.5.7 FlashCopy options for Asynchronous PPRC
There are new options for the FlashCopy V2 Copy Services function available, which are
used mainly in the Asynchronous PPRC environment. Below you can see the new options
available with ESS LIC level 2.4.x or higher for FlashCopy V2:
Establish FlashCopy pair task options:

Fast Reverse Restore

Establish FlashCopy target on existing PPRC Primary

Revertible
Withdraw FlashCopy pair task options:

Revert the FlashCopy

Commit FlashCopy
These options, except for Establish FlashCopy target on existing PPRC Primary, are
used only in an Asynchronous PPRC environment and should not be used for other
purposes. There is also new status information for the FlashCopy volume pairs:

FlashCopy sequence number: This is a unique number that is assigned to all FlashCopy
pairs by the Asynchronous PPRC mechanism every time a new Consistency Group is
created successfully.

Revertible: This is a status flag for the FlashCopy volume pair that tells you if you can
revert the FlashCopy pair to the previous FlashCopy sequence.
Fast Reverse Restore
This FlashCopy V2 option is used to quickly copy the last consistent data from the C volumes
to the B volumes in the Asynchronous PPRC environment. It changes the roles of the
volumes in the FlashCopy relationship. The main usage of this option is in an unplanned
outage scenario in the Asynchronous PPRC environment. Once the background copy
finishes, the FlashCopy relationship will be withdrawn automatically (for more information,
refer to “Fast Reverse Restore” on page 249.)
Note: When using this option to copy the consistent data from the C volumes to the B
volumes (see Figure 4-16) in the Asynchronous PPRC unplanned outage scenario, it is
also necessary to use the Establish FlashCopy target on existing PPRC Primary
option, as the B volumes are usually in the PPRC primary suspended state.

84
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Revertible
This option is used for a two phase Establish FlashCopy pair task. It allows you to prepare
for a new FlashCopy relationship without altering the existing FlashCopy relationship. The
new relationship may be either committed or reverted (restored) by using the Withdraw
FlashCopy pair command with the Revert the FlashCopy or Commit FlashCopy options.
This option is related to the Asynchronous PPRC and is not mainly used in other
environments (for more information on how to use this option, refer to the IBM TotalStorage
Enterprise Storage Server Web Interface User’s Guide, SC26-7448.)
Revert the FlashCopy
This is an option for the Withdraw FlashCopy pair task. It may be used in the Asynchronous
PPRC environment as part of a recovery procedure from an unplanned outage. This task
does not withdraw the FlashCopy pair, it just specifies that you want to use the previous
FlashCopy relationship for the recovery procedure. You can issue this command only if the
status of the FlashCopy pair is Revertible. For more information on how to use this option,
refer to “Revertible” on page 229. In 9.7.3, “Evaluating the FlashCopy status query results” on
page 495, you can find information on when to revert or commit the FlashCopy relationship.
Commit FlashCopy
This is an option for the Withdraw FlashCopy pair task. It may be used in the Asynchronous
PPRC environment as a part of recovery procedure from an unplanned outage. This task
does not withdraw the FlashCopy pair, it just specifies that you want to use the current
FlashCopy relationship for the recovery procedure. For more information on how to use this
option, refer to 6.17.7, “Remote site recovery” on page 354. In 9.7.3, “Evaluating the
FlashCopy status query results” on page 495, you can find information on when to revert or
commit the FlashCopy relationship.
4.6 PPRC data consistency
As mentioned in 4.3.3, “Creating a consistent point-in-time copy” on page 64, data
consistency needs special attention with copy operations. Here we discuss types of
consistency and describe additional ESS features to control consistency.
4.6.1 Consistency types
With respect to operational context and scope, we have to distinguish several types of
consistency. In the context of PPRC operations, there are three types of consistency
guarantees to consider:

No guarantee: This is the weakest type. We are ensured only that given sufficient time
without updates to the primary volumes, the secondary volumes will eventually be
equivalent to the primaries.

Power-failure consistency: This guarantees that the secondary volumes are equivalent
to what would have been seen at the primary site had there been a power failure and
recovery.

Application consistency: This ensures that the secondary volumes are equivalent to
what would have been seen at the primary site had the application been given a chance to
shut down normally.
The first type is our case with PPRC-XD: We know only that all updates will be transferred,
but we cannot make assumptions about the ordering of dependent writes or update
completion time.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
85
In the description of power-failure consistency, you can include other types of hardware
failures, like loss of fibre connections. This is the case with Synchronous PPRC. As noted in
4.3.4, “Synchronous PPRC and PPRC-XD comparison” on page 65, this ensures the
ordering of dependent writes: If the write operation W
1
completes at the primary site before
the write operation W
2
is initiated, then the secondary site never has modifications from W
2

without also having all modifications from W
1
.
Typically, software recovery is required when secondary volumes with this type of
consistency are used. Databases are able to handle that automatically as part of their startup
routine using log files. Because such log files record successful and committed transactions
(any partially committed transactions will be rolled back), this gives transaction-level
consistency.
With power-failure consistency, you still may be exposed to so-called
lost writes
. These are
the “in-flight” transactions that have not been committed. You should expect that
uncommitted transactions will be lost. On the other hand, data that was transferred to the
ESS and confirmed back as written into the NVS (of the secondary ESS in the case of PPRC)
will be destaged to disk.
Application consistency gives the shortest recovery time. But to get an application consistent
copy, you have to quiesce the application’s updates and create a FlashCopy in a catch-up
operation. As discussed in 4.3.4, “Synchronous PPRC and PPRC-XD comparison” on
page 65, this is a point-in time copy that will not be reflective of the current updates, but
rather, any updates up until the last catch-up operation.
With respect to scope, we have to distinguish between volume consistency and global
consistency:

Volume consistency: This means that some type of consistency is guaranteed for the
specific secondary volume.

Global consistency: This means that some type of consistency is guaranteed across all
the volumes upon which the application does updates.
The scope of consistency needed will depend on the application write operations
characteristics, as well as on how the data files are distributed over the production site
volumes.
Synchronous PPRC ensures power-failure global consistency only as long as all volume
pairs are in the full duplex state. When error conditions affect some of the volume pairs (or
different volume pairs at different times), global consistency may be lost. For example, if one
of the secondary volumes cannot be updated because of path failures, then the
corresponding primary volume will normally go into suspended state, still allowing updates.
But these updates are no longer transferred to the secondary volume. Only the bitmap of
changed tracks is created and maintained. So global consistency across volumes is lost,
although the order of writes is still guaranteed for the other secondary volumes.
Consistency groups, as explained below, can help control global consistency.
4.6.2 PPRC Consistency Groups
To maintain consistency of data across volumes at the recovery site, volumes in a PPRC
relationship can be collected into a
PPRC Consistency Group
. A PPRC Consistency Group is
Note: File systems may behave differently, so it is important to perform a file system check
on all PPRC secondary volumes before you start using them.

86
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
a set of volume pairs that have the same primary and secondary LSS for which the PPRC
Consistency Group option has been activated. There are two possible ways to activate this
option:

During the establishment of the path from the primary to the secondary LSS (see 6.12,
“PPRC setup examples using the Web User Interface” on page 259). You select the check
box PPRC Consistency Group in the path options panel.
If the option is activated in that way, then all volume pairs in a PPRC relationship from this
primary to this secondary LSS (sharing the same paths) belong to a Consistency Group.
Other volumes pairs are not affected.

At the LSS level. You select the check box PPRC Consistency Group in the LSS
properties panel.
If the option is activated in that way, then all volume pairs in a PPRC relationship from this
primary to any secondary LSS (using any path) are affected.
This attribute changes the behavior of these volume pairs when an error occurs that affects
any of these volumes:

Without the PPRC Consistency Group option, the ESS causes the volume where the
error is detected to enter a suspended state, but still allows updates to that volume.

If the PPRC Consistency Group option was activated, the volume becomes suspended
and additionally enters a long busy state. Updates are not possible in that state.
The
long busy state
is defined for System/390® as a unit check with a special code that tells
the host error recovery program to stop the current I/O operation and requeue it until the
control unit reconnects or two minutes have passed. Then the I/O will be redriven. If the
control unit is still busy, the unit check is repeated.
For open systems (attached via SCSI or FCP), this state is presented as a
QUEUE FULL
(QF)
status byte code to the host. A status byte is sent during the STATUS phase at the
completion of each SCSI command (unless the command is terminated by a special event).
The QF status signals to the host that the tagged SCSI command queue is full and that the
I/O request has not been placed in the queue. Then the host system will attempt to requeue
requests in the logical unit’s device queue. So Consistency Grouping of volumes provides the
ability to temporarily queue (at the host’s level) subsequent write operations to all PPRC
Consistency Group volumes on a single LSS pairing when an error occurs to one of the
volumes in the group (primary or secondary), or when a total link failure is detected between
the primary and secondary LSS pair (see Figure 4-17 on page 88).
Details of QF handling may be modified by the operating system’s and the host bus adapter’s
settings. For example, in some UNIX-type systems, the utility
scsi_ifsetcap
provides options
to enable or disable SCSI command retry when a QF status is returned. Some HBA tools
allow similar modifications. For example, the Emulex FC Port Driver Utility for Windows has a
setting to translate QF to BUSY to work around a behavior of the disk class driver. You should
check that the settings on your host systems are according to the IBM TotalStorage
Enterprise Storage Server Host System Attachment Guide, SC26-7446.
Note: With PPRC-XD, there is no need to enable the PPRC Consistency Group option.
PPRC-XD always keeps a fuzzy copy at the recovery site. PPRC-XD pairs become
consistent only when all updates are transmitted to the secondary by catch-up while the
application writes are quiesced. When building a consistent point-in-time copy on the
PPRC-XD secondaries, you will need to have the application writes already quiesced,
which makes the QF time-out window unnecessary.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
87
When the first copy pair in the Consistency Group becomes suspended and enters QF, the
ESS error recovery program (ERP) issues an SNMP notification (trap 200). When
subsequent pairs in the Consistency Group become suspended (during a total path failure),
the information is recorded in a table. As other pairs suspend, they are added to the table.
Five seconds after the second pair is suspended, the ERP issues an SNMP notification
(trap 202) listing the further suspended pairs.
An automation program, which is triggered by the notification during the QF interval on the
erroneous copy pairs, can issue a
freeze
(refer to “Freeze operation” on page 90) to all LSS
pairs relating to the application, affecting the other set of primary volumes in the Consistency
Group. The freeze operation causes the primary volumes to suspend, and go into QF, as well
as terminating their associated paths. Therefore, all updates at the recovery site are halted,
resulting in a consistent checkpoint. Notice that such an automation program is not part of the
ESS software and must be supplied by the user.
The Consistency Grouping timer function gives the automation software time to issue the
automated commands. The QF time-out value can be displayed and changed from the
Logical Subsystems panel of the ESS Copy Services Web User Interface by selecting the
desired LSS and then clicking the Properties button. The Logical Subsystems Properties
panel is displayed, and there in the Consistency Group Time Out field, you can check or
modify the time-out value (refer to 4.6, “PPRC data consistency” on page 84 and 6.6.4, “LSS
properties” on page 212). The default value for this parameter is set to two minutes.
Attention: It is important to notice that the QUEUE FULL condition is presented only for
that primary volume that is affected by the error (in the case of path failures, multiple
volumes are often affected). But a freeze operation is performed at the LSS level, causing
all volumes in that LSS to go into suspended state with QF condition and terminating all
associated paths.
Therefore, do not intermix volumes relating to different applications in an LSS pairing that
is part of a Consistency Group, because the non-related volumes will be frozen too.

88
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 4-17 Error sequence on Consistency Groups
The following examples illustrate the contents of trap 200 and 202 issued to the automation
program by the ESS ERP.
Example 4-1 illustrates that the PPRC pair with LUN ID 0 in LSS 2 in the Consistency Group
is in error.
Example 4-1 SNMP trap 200
1002: 2002/09/05 13:51:32 CDT
LSS-Pair Consistency Group PPRC-Pair Error
UNIT: Mnf Type-Mod SerialNm LS LD SR
PRI: IBM 2105-800 13-ABC12 2 0 8
SEC: IBM 2105-800 75-DEF34 2 0
Example 4-2 shows that the PPRC pair with LUN ID 2B in LSS 14 has become suspended.
Example 4-2 SNMP trap 202
2002/08/23 16:34:02 CDT
Primary PPRC Devices on LSS Suspended Due to Error
UNIT: Mnf Type-Mod SerialNm LS LD SR
PRI: IBM 2105-800 13-ABC12 14 2B 09
SEC: IBM 2105-800 75-DEF34 14 2B
Start: 2002/08/23 16:33:57 CDT
PRI Dev Flags (1 bit/Dev, 1=Suspended):
FFF0000000100000000000000000000000000000000000000000000000000000
Figure 4-18 on page 89 provides a summary of Consistency Groups.
Application
Server
NetView
Console
X
Queue Full
1.Error detected, trap 200
raised.
2.Primary volume in error is
suspended and goes into
Queue Full condition.
3.Other volumes unaffected.
4.Record suspends of
subsequent pairs.
I/O
Application
Server
NetView
Console
Queue Full
Trap-202
1.Automation issues freeze to
Consistency Group.
2.All volumes in Consistency
Group suspended.
3.All volumes in consistency
group go into Queue Full.
4.All paths for the Consistency
Group terminate.
5.Five seconds after the
second pair suspends, trap
202 is raised listing the
subsequent suspended
volumes.
FREEZE
Trap-200
202.
volume
001 sus.
002 sus.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
89
Figure 4-18 Consistency group summary
CRIT attribute
The CRIT attribute is used to determine the behavior of the PPRC pairs or Consistency
Groups after a failure in communication between the primary and secondary ESSs, when all
the paths between a pair are lost. It is used to maintain the consistency of the replicated data.
The ability to use the CRIT parameter is set in the ESS Vital Product Data (VPD). In the event
that this parameter needs to be altered, a power cycle of the ESS is necessary. The use of
this parameter is set using ESS Web CopyServices or through TSO commands.
CRIT (NO) specifies that following an I/O completion error to the secondary volume, PPRC
suspends the copy pair, allowing subsequent write requests to the PPRC primary volume to
be satisfied. All writes subsequent to the suspension of the copy pair are recorded for update
on the resolution of the problem. This is the default setting.
CRIT (YES) specifies that if an I/O error to the secondary volume occurs, PPRC either allows
or does not allow subsequent writes to the primary, depending on how the storage subsystem
is configured. The PPRC pair then remains in a suspended state until you correct the problem
and either issue a command to resynchronize the PPRC pair or delete the PPRC pair.
Note: This parameter only applies to System/390 volumes. This section is included here
for the sake of completeness.
Consistency grouping:
Enables the Queue-Full (QF) condition.
Activated.
When defining the paths between primary and secondary LSSs or as LSS property (path
independent).
Triggers the QF condition for SYNC pairs when suspended because of errors.
Queue Full:
Errors in a volume pair result in the LSS pair being suspended and the QF instigated.
While in QF, all write operations to the affected primary LSS are temporarily queued.
Meanwhile, PPRC sends alert messages to host operating system for the suspended LSS
pair.
The alerts can be used to trigger automated operations to create a point of consistency.
QF timeout (by default, two minutes) causes the QF to end, and the writes are resumed.
The thaw operation allows writes to resume before QF time-out expires.
QF time-out can be modified from the Logical Subsystems panel when
using the ESS Copy Services Web User Interface.
Consistency grouping is valid to create point of consistency when
volumes are in SYNC mirroring. Not necessary when volumes are in
PPRC-XD relationship.
PPRC-XD always keeps fuzzy copy at the recovery site.
PPRC-XD pairs become consistent only when all updates are transmitted to secondary by
catch-up while application writes are quiesced.
Gives time for multiple freeze operations in order to reach global consistency.

90
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The implementation of CRIT (YES) on the ESS is similar to the implementation on the
IBM 3990. There is an option that can be set by the CE in the VPD of the ESS that
determines how this CRIT(YES) setting will behave in an error situation:

CRIT=YES - Paths (light version)
– Suspend the pair and do not accept any further writes if the control units can no longer
communicate.
– Suspend the pair and accept further writes if the control units still can communicate
with each other. The reason for not being able to copy the data to the remote volume is
probably only a device problem on the secondary site and not a disaster. Therefore, we
continue with write operations to the primary volume. The ESS records which cylinders
have changed. After investigating the problem and after it has been solved, you can
re-synchronize the source and target volume again.

CRIT=YES - All (heavy version)
– Suspend the pair and do not accept any further writes to the primary volume if data
cannot be sent to the secondary volume.
Freeze operation
With a
freeze
operation, you can stop the write activity on all the active PPRC primary and
secondary volumes of a given source and target LSS pair. This function enables you to
maintain secondary volume update consistency. It affects all the volumes in the Consistency
Group that are in a PPRC active copy process: duplex, duplex pending synchronous, or
duplex pending PPRC-XD states. It does not affect the suspended and simplex volumes or
volumes that are not part of the Consistency Group.
The freeze function operates on LSS pairs that have been defined with (or without) the PPRC
Consistency Group option enabled. The freeze operation has three effects:

The paths that connect the pair of LSSs being frozen are terminated.

The active Consistency Group volumes under the frozen LSS pair are suspended. This
state transition (to suspended) is then communicated to the host with SNMP alert
messages. These alert messages can be used by automation routines to trigger other
recovery operations.

If the PPRC Consistency Group option was enabled at path definition time or in the LSS
properties panel, then, additionally, the QF condition is instigated, so the write activity to
the primary LSS is temporarily queued. During this QF interval, other automated
operations can be triggered, for example, freezing other application-related LSS pairs.
When several freeze operations are executed within the QF time-out interval for all the
application-related LSS pairs, then you can insure a global consistent checkpoint at the
recovery site. For this to be possible, you should have defined all these related LSS pairs
associations with the PPRC Consistency Group option.
When you suspend a pair of volumes, whether specifying the source or the target LSS, you
will observe that both volumes of the pair appear as suspended after the operation. Both
volumes are still linked by existing logical paths. But this is not the case when you do a freeze
operation. For this situation, only the primary volumes show as suspended. The secondary
volumes will appear as an active secondary.
When freezing volumes in a duplex state, the secondaries will be a consistent copy. When
freezing volumes in duplex pending PPRC-XD or duplex pending synchronous state, the
secondaries will be in an indeterminate point of consistency.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
91
Once the error has been resolved, the Consistency Group enabled LSSs are
thawed
to
remove the resulting QF condition, and to reset the suspended volume count table. The thaw
operation is started when you run a Consistency Group created task at the LSS level. This
operation closes the time-out window, and allows writes to the primary volumes to resume.
However, the affected pairs remain suspended, and the paths will not be automatically
re-established. This has to be done manually, so it is advisable to create and save these
tasks.
Before re-establishing the paths, in order to reestablish the pairs, you may perform a
FlashCopy of the secondary volumes, because at this moment they are holding a globally
consistent point-in-time copy of the data. Figure 4-19 summarizes the characteristics of the
freeze operation.
Figure 4-19 Freeze and resume summary
4.7 Planning for PPRC
For using PPRC on an Enterprise Storage Server, some requirements must be fulfilled.
Besides the planning of volumes and Copy Services tasks, you have to check the following
items:
PPRC hardware requirements
PPRC is only supported between the same type of storage subsystems. Therefore, it is only
possible for an Enterprise Storage Server to perform PPRC replication to another Enterprise
Storage Server. It is possible to carry out PPRC between different models of the ESS. An
ESS F20 can be in a PPRC relationship with an ESS 800, for example.
You will need to purchase ESCON or Fibre Channel adapters and cables, possibly with
additional equipment, depending on the distance between the primary and the secondary
ESSs. ESCON or Fibre Channel connections have to be configured between the primary and
The freeze operation works on an LSS basis:
Duplex, duplex pending PPRC-XD, and duplex pending SYNC volumes are frozen.
Non-PPRC (simplex) and suspended pairs in the same LSS are not frozen.
PPRC pairs not part of the consistency in the same LSS remain unaffected.
The primary LSS stops propagation of updates to the secondary LSS:
Removes the paths between the primary and secondary LSS pair.
Suspends the volumes pairs.
Secondary volumes that were in duplex state will have a consistent point of data.
Secondary volumes that were in duplex pending XD or duplex pending SYNC will have an
indeterminate consistency.
If LSSs were consistency group enabled:
The LSS is put in the QF condition when the first write comes.
The QF prevents updates to the primary volumes during the default two minute time-out
interval. Updates are temporarily queued.
The time-out interval will end if consistency-group-created task (thaw) is invoked.
PPRC pairs remain suspended.
The time-out interval value (two minutes) can be changed with the Web user interface.
To resume the mirroring activity to the secondary volumes:
The paths must be re-established.
The pairs must be re-established.

92
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
secondary units over which the replication will occur. Figure 4-20 on page 94 illustrates a
Copy Services configuration.
PPRC software requirements
PPRC is a part of Copy Services, which is a combination of software and licensed internal
code that runs on each ESS cluster in a Copy Services Domain.
Although PPRC is part of the ESS-internal software, it requires you to purchase a usage
license. It needs to be activated on the Enterprise Storage Server clusters that will be
participating, using an activation key applied by the IBM customer engineer (IBM CE).
PPRC is an optional chargeable feature of the ESS product and is available in two versions.
PPRC Version 1 provides real-time mirroring and extended distance:

Synchronous PPRC

PPRC-XD
PPRC Version 2 further supports business continuance solutions with the following additional
functions:

PPRC over Fibre Channel links

Asynchronous PPRC

Asynchronous Cascading PPRC
Note that PPRC Version 2 includes all PPRC Version 1 features. Only one of the PPRC
features (V1 or V2) can be ordered on the same machine. Also, any other ESS that will
contain the PPRC pairs must also have the minimum PPRC feature version required to
support the type of Copy Services being used. Also note that FlashCopy Version 1 cannot be
ordered with PPRC Version 2 on the same machine.
Table 4-4 shows all prerequisites for various PPRC functions.
Table 4-4 Minimum requirements for PPRC features
Note that PPRC V2 includes all PPRC V1 features. Only one of the PPRC features (V1 or V2)
can be enabled on the same machine. Also, any other ESS that will contain the PPRC pairs
must also have the minimum PPRC feature version required to support the type of Copy
Services being used. Also note that FlashCopy V1 cannot be ordered with PPRC V2 on the
same machine.
For more details, refer to Appendix B, “Copy Services feature codes” on page 541.
PPRC Feature Minimum PPRC
License
Minimum ESS Licensed Internal Code
F
XX
800 750
Synchronous PPRC PPRC-V1 1.3.0 2.0.0 2.3.1
PPRC-XD PPRC-V1 1.5.2 2.0.0 2.3.1
Asynchronous Cascading PPRC PPRC-V2 2.2.0 2.2.0 2.3.1
Asynchronous PPRC PPRC-V2 no support 2.4.0 2.4.0
PPRC over ESCON PPRC-V1 1.3.0 2.0.0 2.3.1
PPRC over Fibre Channel PPRC-V2 no support 2.3.0 2.3.1

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
93
Network requirements
All the ESS clusters should be configured to communicate over Ethernet using TCP/IP. Each
cluster within each ESS will have their own TCP/IP network address.
The Copy Services servers and their client clusters need to have their IP addresses
resolvable, either using DNS or a standard /etc/hosts file during installation. The method for
IP-Address resolution can be configured by the IBM CE during installation.
Optional SNMP configuration
The ESS has the ability to raise SNMP alerts when an error condition has been encountered.
When an error is encountered, an alert is posted to a systems management console, such as
Tivoli Netview. During installation, the TCP/IP network address of the systems management
console, along with the community to which the ESS alerts will be posted, will need to be
configured. This becomes more pertinent for capturing the new traps that are raised during
ESCON or FCP link loss (see 4.9.4, “PPRC path failure alerts” on page 107) and errors on
volumes belonging to
Consistency Groups
(see 4.6, “PPRC data consistency” on page 84). It
is the systems management console that will perform the automation on the receipt of an
alert.
Management stations
You have to plan which machines will be used to control PPRC operations. These machines
must be connected to the ESS through the IBM TotalStorage Enterprise Storage Server
Network (
ESSNet
). The ESSNet is the ESS access facility that IBM installs when they install
your ESS, consisting of a switch (hub) and other networking components and the dedicated
IBM TotalStorage Enterprise Storage Server Master Console (
ESS Master Console
).
With respect to Disaster Recovery, you should plan for at least two management stations (one
in each site) that do not depend on the ESS for proper operation.
Web browser
A browser for the ESS Specialist has to be installed on the management stations. For
recommendations about Web browser versions, see IBM TotalStorage Enterprise Storge
Server Web Interface User’s Guide, SC26-7448.
ESS Copy Services CLI
If you plan to use the CLI, you have to install the CLI also on the management stations. For
details, see the IBM TotalStorage Enterprise Storage Server Command-Line Interfaces
User’s Guide, SC26-7494.
Task structure
To avoid human mistakes and automate routine actions, you will create and save tasks in
advance. We highly recommend that you use an extendable task naming schema (see
Appendix E, “Sample naming convention” on page 561). Prepare tasks for PPRC
establishment with several options. Table 4-5 on page 94 summarizes the combinations of
options available for establishing PPRC relationships. Also, prepare the corresponding tasks
to suspend and terminate the PPRC relationships to be scheduled to the primary as well as to
the secondary LSS.

94
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Table 4-5 Task options for establishing PPRC relations
When planning resources for your PPRC environment (see Figure 4-20), you need to
consider which data is critical and requires protection in the event of a disaster. Knowing this,
it is possible to plan the throughput and capacity requirement of the secondary ESS. It is
important to realize that the capacity needed on the secondary ESS for Disaster Recovery
may not have to be initially as large as the primary ESS. Some applications and data will be
more critical than others. You need to size your secondary ESS based on your critical
business requirements, possibly with some headroom for applications of intermediate
importance.
Figure 4-20 ESS Copy Services configuration
Establish PPRC Permit read from
secondary
Suspend PPRC after
establish completes
Asynchronous
Cascading PPRC
Copy entire volume SYNC + XD SYNC only SYNC + XD
Do not copy volume SYNC + XD Not available SYNC + XD
Copy out-of-sync
cylinders only
SYNC + XD SYNC only SYNC + XD
PPRC Failover Not available Not available SYNC + XD
1
PPRC Failback SYNC + XD SYNC only SYNC + XD
Notes: SYNC and XD mean that the combination is available for Synchronous or XD mode. The
options Critical volume mode and Permit establish if target is online are not available for
open-systems volumes.
1)
PPRC-XD Failover available with LIC 2.3.x
Copy Services
Server B
Ethernet
Hub
Copy Services
Server A
Copy Services
Client
Up to eight ESCON
paths between LSSs
Ethernet LAN
connections
Up to four
secondary
LSSs
Application
Server
StorWatch
Specialist
Copy Services
Client
Up to n
primary
LSSs
Up to n physical ESCON
links between ESSs

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
95
4.8 Choosing the best method
Each of the PPRC Copy Services features has different characteristics designed to cover a
wide range of client needs. Every business has it’s own special and unique requirements and
within a business there is usually a diverse range of applications. Some applications may
require fast recovery due to critical online requirements, others cannot tolerate the loss of a
single transaction.
For each different business application, the storage management requirements are unique,
but can generally be categorized under the following general points (examples given to
illustrate):

Data Loss (the amount of data that is lost after recovery)
– Transaction based applications, financials, ordering. Little or no data loss tolerable.
– Database applications with logging. Some loss tolerable.
– Batch. Application can be rerun, data loss can be recovered.

Consistency (writes to all volumes are in the correct time sequence)
– Database applications: Each transaction is logged when started and again when
completed to allow correct recovery. Log and data must be in step.
– Preserves the order of dependent writes.
– Operating systems: File system recovery (journaling or VTOC consistency), catalogs.

Outage time (time to recover)
– In the event of a disaster or failure, the time it will take to bring backup systems up on
the mirrored volumes.
– Businesses with customer facing online systems may not be able to tolerate more than
minutes of down time.
– Most online database systems are required to be back online within an hour.
– Many businesses, using risk management of non critical processes, accept recovery
times in hours or days depending on the application requirements.

Distance between production and recovery sites
– Same site.
– Recovery or backup site in same city, or nearby - Metro, 0 - 300 km.
– Another city, country or continent - Global, greater than 300 km.

Copying requirements
– Point-in-time mirrored copies for disaster recovery testing, Quality Assurance testing,
and archive backups.
– Single volume mirrored copies, locally or at another site.
– Multiple volume mirrored copies.
– Continuous copies of data from a live system without disruption.
While recovery remains with the systems and the applications, the use of the appropriate
Copy Services’ feature will ensure that the data is copied in a suitable state for recovery
depending on the business and application requirement.
There are four PPRC features designed to meet the different needs of client’s businesses
and, within their enterprise, the different needs of individual applications.

Synchronous PPRC: Metro, zero data loss

96
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

PPRC-XD: Global

Asynchronous Cascading PPRC: Global, zero data loss

Asynchronous PPRC: Global, point-in-time consistent
4.8.1 Comparing PPRC features
In Figure 4-21, we show a comparison of the main attributes of each PPRC feature. The
following discussion explains this diagram in more detail.
Figure 4-21 PPRC comparisons
Terms

Local I/O response is the response time to the host I/O at the local or primary site from I/O
start to completion notification. A low value is a short (good) response time.

Level of consistency is the degree of consistency of the data after the copy operation and
recovery. The data across all volumes is in sequence and the last writes were at the same
point-in-time. A high value indicates good consistency.

Data loss refers to the amount of data that was written at the primary site after the copy
operation or that cannot be recovered at the remote site. A low value is a small amount of
data loss.
Note that these graphs are only for comparison and are not to scale.
L
o
c
a
l

I
/
O

R
e
s
p
o
n
s
e
L
e
v
e
l
o
f







C
o
n
s
i
s
t
e
n
c
y

Ti
m
e

t
o

R
e
c
o
v
e
r
y
D
a
t
a

L
o
s
s
C
o
s
t
Synchronous PPRC - Short Distance
Synchronous PPRC - Long Distance
PPRC-XD
Asynchronous Cascading PPRC
Asynchronous PPRC

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
97

Cost: Comparative cost of ESS resources (capacity, ESS subsystems), site resources,
and recovery workload. Higher value, higher cost.

Time to recovery is the comparative time it takes from a disaster event occurring to having
only the data ready to use at the recovery site. This does not represent the recovery of the
entire application and all business processes. A low value is a short time.
Refer to the diagram in Figure 4-21 on page 96 for the following discussions.
Synchronous PPRC - Short Distance
Consider short distances as meters up to 20 kilometers. The remote ESS may be in the same
building, but will most likely be at another site in the same city. Typical links would be via
DWDM or SAN.
Local I/O response
The response time for the I/Os will be very good, but noticeably higher than running without
PPRC. This is usually acceptable. I/Os are accepted, written to the local ESS and also sent to
the remote. Successful I/O completion is not returned to the local host until the remote has
responded that the write operation is also complete.
Consistency
This mode of operation gives continuous consistency. As completion of the write operation is
not reported to the host until the remote site also has written the data, then data is always
consistent in terms of the applications.
Data loss
After return of I/O completion to the application, the data at the remote site is the same as the
data at the local site. From this point of view, there is no data loss. Incomplete operations may
have to be discarded or recovered. There may be some further application data loss after the
recovery of the file systems or data base recovery, for example, in-flight transactions.
Cost
For full coverage, a second site is required. This will require the installation of ESS
subsystems at the remote site with sufficient capacity to create a volume for every volume at
the primary site that needs to be mirrored. ESCON or SAN connectivity between the sites is
also required with sufficient bandwidth to accommodate all traffic at peak loads.
Time to recovery
As the remote data is always consistent, recovery of the storage is very fast. As soon as the
PPRC relationships have been suspended (which can usually be done with FREEZE
commands), the data can be accessed and used at the remote site.
Applications

Critical online systems

Financial and legal transaction processing

Customer facing online systems

Copying

Site failover requirements

Entire system mirroring
Reference: Synchronous PPRC is discussed in 4.2, “Synchronous PPRC” on page 57.

98
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Synchronous PPRC - Long Distance
The same as the short distance description above, only over local or metro distances,
considered to be in the same city or nearby. The behavior is much the same, only the local
response time is adversely affected as the distance increases.
Local I/O response
As the distance increases, the response time to I/O at the local sites increases proportionally.
For continuous operation, a maximum distance of 103 km over ESCON and 300 km over
FCP is supported. It is possible to use synchronous mode over longer distances during low or
zero I/O rates as part of recovery using one of the non-synchronous modes.
This distance factor needs to be carefully considered, as the degradation in response time
may exceed application tolerance long before the maximum distance is reached. Your IBM
Representative will be able to assist with planning, but it ultimately depends on your business
requirements and applications.
Consistency
As for short distance.
Data loss
As for short distance.
Cost
Because the distance is increasing, so also does the cost. For example, Channel Extenders
or DWDMs add to the cost. It is necessary to maintain the PPRC links continuously with
sufficient bandwidth to cope with the maximum workload.
Time to recovery
As for short distance.
Applications

As above, where geographic separation of sites is required within metro distances.
PPRC-XD
Designed for global distances, the effect on local response times is negligible, while
asynchronously mirroring to the remote site. Very good as part of a solution to copy data
remotely with little impact, to maintain an almost up-to-date copy of data or as used in
conjunction with other ESS Copy Services features.
Local I/O response
This is excellent and almost the same as running native without PPRC. Because the write to
the remote ESS is batched and sent asynchronously, there is no impact to the host. I/O
complete is returned as soon as the write is complete in the local ESS.
Consistency
Poor. As the load on the paths increases the I/O transfers can become more out-of-sync
between volumes. When the I/O rate decreases, the copy process is able to catch up, so the
level of consistency will vary depending on load.
Reference: Synchronous PPRC is discussed in 4.2, “Synchronous PPRC” on page 57.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
99
Provided applications are able to recover from a checkpoint, then this still provides a very
useful and low impact solution. A good solution for databases is to send the data via
PPRC-XD and send the transaction logs with Synchronous PPRC.
If the primary site ESS is still available, then there is the ability to go-to-sync. This changes
the PPRC-XD relationship to Synchronous PPRC mode. There will be a delay while the
remote copies catch up, then the rules are the same as for Synchronous PPRC above. Also
included in this feature is the ability to freeze the primary site, which creates consistency
across multiple volumes.
Data loss
Depending on application checkpoints, this can be high. Generally, applications will have to
go back to a known good point and recover forward. The amount of data loss will depend on
the level of recovery, but will generally be worse than other methods of PPRC.
Cost
ESS resources are the same as with Synchronous PPRC, although there is usually a need to
have volumes available for making FlashCopies at the remote site. Path requirements are
less for two main reasons:
1.We only have to allow for the average throughput. This also needs to be carefully sized to
ensure we do not get too far behind.
2.Because the PPRC process is batching the copies to the remote, there is more efficient
use of the available bandwidth.
Time to recovery
This can be very long, depending on the application’s ability to recover. Database recovery
and rebuild can take many hours or may not be possible, requiring backup restore. It may be
required to go back to a checkpoint and migrate forward to a point close to the failure. For
some applications where writes are not frequent (such as web hosting), recovery time may be
short and systems can be brought online at the remote site reasonably quickly. Generally, the
recovery time from this inconsistent state should be considered unpredictable.
Applications

Low impact database and batch recovery.

Copying to test site for testing or planned maintenance.

Low cost solution for global distance requirements where inconsistency and lag can be
tolerated.
Asynchronous Cascading PPRC
A very good solution where data is mirrored synchronously to a nearby intermediate site, then
asynchronously at global distances to the remote site. This gives full protection against loss of
any one of the three sites, while still maintaining good response times at the primary site and
zero RPO with no data loss at the remote site. Also very fast recovery times.
Local I/O response
Because the first link of the PPRC copy is synchronous, this will depend on the distance. Use
the same rules as Synchronous PPRC to determine how far away the
bunker
(intermediate)
site can be and still meet the response time requirements of the applications.
Reference: PPRC-XD is discussed in 4.3, “PPRC Extended Distance (PPRC-XD)” on
page 61.

100
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Consistency
Again, because the second link is running PPRC-XD, the consistency while running is poor.
But, with the use of an intermediate site (and assuming the risk of losing the primary and
intermediate sites simultaneously is very low) the data can later be caught up, even after the
primary site has failed. By issuing commands to go-to-sync or simply waiting for the
PPRC-XD to catch-up, then this gives excellent consistency at the remote site.
Depending on the implementation, there may be varying degrees of risk when using this
method. If a two-site implementation is used, then the risk increases.
Data loss
Again, provided there is only one site lost in a three-site implementation, then there is no data
loss and the recovery is the same as for Synchronous PPRC.
Cost
The need for an additional site and replication of the equipment of the remote site at the
intermediate site makes this the most costly solution. Also the need for two sets of links
between sites. This can be done with two sites, but this compromises the protection against
primary site loss.
Time to recovery
Time to recovery is very good. Once the PPRC-XD on the second link has drained all the
changes (or been forced to synchronous mode), then the data at the recovery site is ready for
use.
Applications

Critical applications requiring quick recovery times and near zero data loss over global
distances.
Asynchronous PPRC
Continuously updated Consistency Groups are being created at the remote site on an
additional set of volumes. These are managed and controlled by the
Master
, running in one of
the ESSs in the enterprise. This PPRC relationship consists of two parts, a PPRC-XD pair
that can be over global distances and a FlashCopy pair on the remote site used to make
consistent copies.
Local I/O response
Because we are using PPRC-XD, this is excellent, with negligible difference from native
non-PPRC writes.
Consistency
Excellent consistency. The FlashCopies at the remote site are synchronized in time with the
data; therefore, the order of dependent writes is preserved. Another Consistency Group is not
started unless the previous one completed or timed out. If the FlashCopy is not completed on
all volumes, there is the ability to revert to the previous FlashCopy.
Data loss
The creation of Consistency Groups can be on the order of seconds apart. Five seconds and
less is quite achievable. Tuning and application requirements (discussed later in
“Start/Resume Async PPRC” on page 80) will dictate the best values, part of which are user
Reference: Asynchronous Cascading PPRC is discussed in 4.5, “Asynchronous PPRC”
on page 74.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
101
setable. Also the available bandwidth is a factor. The recovery process is to recover to the last
successful FlashCopy.
Cost
Requires the remote site and additional ESS hardware with capacity to mirror every volume
that needs to be mirrored times two: one for the PPRC target and one for the FlashCopy.
Time to recovery
The consistency of the FlashCopies must be assessed and then the data recovered back to
the remote volumes for use by the applications. IBM has tools available to automate and
manage this process. (See 9.9, “Multiple Device Manager Replication Manager” on page 505
and 9.10, “Asynchronous PPRC Utilities for Open System Environment” on page 507.)
Applications

Most high availability business applications where a minimum data loss can be tolerated.

Systems requiring fast recovery at the remote recovery site.

Global distances.
4.9 PPRC connectivity
When implementing a PPRC configuration, one requirement is the definition of the paths that
PPRC is going use to communicate between the primary and secondary LSSs. These logical
paths are defined upon existing physical links. The physical links can either be ESCON links
or Fibre Channel links.
In this section, we discuss the physical connectivity and the logical definitions of paths
between the primary and secondary ESSs.
4.9.1 ESCON links
ESCON (Enterprise Systems Connection) is an IBM architecture with fiber optic technology.
ESCON was the first implementation of storage networking before there was a Fibre Channel
standard. Introduced in 1989, ESCON is a 200 Megabit per second (Mb/sec.) serial protocol
that can achieve effective instantaneous data transfer rates up to 18 MB/sec. ESCON is used
to connect S/390 and later generation IBM
mainframe
computers to peripherals such as disk
systems. It allows a data center installation to replace electrical-copper cables with fiber optic
cables. Beyond this physical aspect, ESCON provides the capabilities to connect ESS units
at distances beyond 103 km. Similar to Fibre Channel, ESCON is based on message
exchanges.
An optical fiber functions as a kind of wave guide for light. It is usually made of silica glass.
The fiber itself has a central core and a surrounding cladding of slightly different glass
material. The physical size of an optical fiber is determined by the diameter of the core and
cladding, expressed in micrometers (µm). A fiber optic cable having a core diameter of
62.5 µm and a cladding of 125 µm is designated as 62.5/125 µm optical fiber. Other fibers
Reference: Asynchronous PPRC is discussed in 4.5, “Asynchronous PPRC” on page 74.
Note: The present section discusses PPRC links and paths when using ESCON links.
Because many of the discussions are valid to both ESCON and Fibre Channel links, we
recommend that you first read the present section and then proceed to read 4.10, “PPRC
using Fibre Channel links” on page 111 for the Fibre Channel specifics.

102
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
commonly used for ESCON are 50/125 µm and 9/125 µm. The ESCON cables consist of two
fibers.
Two modes can be used to send light signals through an optical fiber: single mode (mono
mode) or multimode. The optical fibers used are called accordingly single-mode or multimode
fibers. These fibers have different physical dimensions and light transmission characteristics.
The multimode fibers supported by IBM are either 62.5/125 µm or 50/125 µm. The light
source used for multimode fiber is usually a light emitting diode (LED). The LED jumpers are
orange, and their duplex connectors are black. The maximum distance for a multimode fiber
link is 3 km if 62.5/125 µm fibers are used and 2 km if 50/125 µm fibers are used. The
maximum distance for a multimode fiber link is 2 km if both 62.5/125 µm and 50/125 µm
fibers are used.
Single-mode fiber usually has a core diameter of 8 to 10 µm and a cladding diameter of
125 µm. The light source used for single-mode fiber is a laser. IBM supports single-mode
9/125 µm fibers for use in an ESCON environment. Single-mode fiber must be used on links
between devices with ESCON-Extended Distance Feature (XDF) adapters installed. These
adapters are available only on ESCON Directors and on remote channel extenders.
Fiber jumper cables of 9/125 µm for IBM ESCON Directors are supplied by IBM in standard
lengths up to 122 m (400 ft.), if XDF ports are ordered. The number of jumper cables supplied
at no charge depends on the number of XDF ports installed. XDF jumpers are yellow, and
their duplex connectors are grey. The maximum distance for a single-mode fiber link is
20 km.
The number of ESCON links between the primary and secondary units is dependent on the
performance requirements and the availability of slots in the hostbays (possibility of
32 ESCON links). ESCON channels provide 160 Mbps point-to-point theoretical throughput.
The ESCON host adapter ports have the capability of operating in two ways, but when
implemented for PPRC transmission, they operate in either one of those two ways:

The ESCON host adapter port operates in
control unit
mode, when it is talking to a host.
In this mode, an ESCON port can also receive data from a primary ESS, when the ESS
port is connected to an ESCON director.

An ESCON port is operating in
channel mode
, when it is used on the primary ESS for
PPRC operations onto a secondary ESS.
Therefore, ESCON links for PPRC are unidirectional. The primary unit ESCON port (the one
in channel mode) has to be dedicated for PPRC. The ESCON port on the secondary unit can
also be used for S/390 host attachment, provided the ESCON director is used, and the host is
connected to it. This is illustrated in Figure 4-22 on page 103.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
103
Figure 4-22 PPRC ESCON connectivity
Direct ESCON connection
Direct ESCON link connections between primary and secondary ESS units can be
established without the need of any intermediary devices. Standard ESCON adapters are
used in the ESS; this connection can only be a multimode connection. Direct connections can
be established for up to 2 km with 50/125 µm multimode fiber ESCON cables and 3 km with
62.5/125 µm multimode fiber ESCON cables. The ESCON channel performance is a function
of distance. The direct connection is shown in Figure 4-23.
Figure 4-23 Point-to-Point PPRC configuration
ESCON Directors
The distance between primary and secondary ESS units can be extended using an ESCON
Director as a signal relay. The maximum distance between each ESS and the ESCON
Director can be 2 km (50/125 µm fiber) or 3 km (62.5/125 µm fiber), resulting in a maximum
distance between both ESSs of 4 km or 6 km, respectively (see Figure 4-24 on page 104).
Port A
Port B
Port A
Port B
Port A
Port B
Port A
Port B
ESCON
Adapter
ESCON
Adapter
ESCON
Adapter
ESCON
Adapter
Dedicated
Dedicated
Dedicated
Primary
ESCON
port is
dedicated
PPRC links are uni-directional
to host
to host
ESS
ESS
Shared
Secondary ESS
TotalStorage
Primary ESS
TotalStorage
PPRC ESCON

104
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 4-24 Configuration with one ESCON director
Adding a second ESCON Director is the next step to increasing the distance. The distance
between the two ESCON Directors can be 2 km for 50/125 µm fiber and 3 km for
62.5./125 µm fiber.
You can use Extended Distance Feature (
XDF
), which installs ESCON-XDF adapter ports on
the ESCON Directors with 9/125 µm single mode fiber optic cables. This allows the maximum
distance between the XDF Director ports to extend to 20 km, providing an overall distance of
26 km between ESS units. This can be seen in Figure 4-25.
Figure 4-25 Configuration with two ESCON directors
Channel extenders
Various channel extender vendors can also be used to increase the distance between ESS
servers via a Wide Area Network (
WAN
) using a variety of connections, such as Fibre
Channel, Ethernet/IP, ATM-OC3, and T1/T3.
When using channel extender products with PPRC, the channel extender vendor will
determine the maximum distance supported between the primary and secondary ESS. The
channel extender vendor should be contacted for their distance capability, line quality
requirements, and WAN attachment capabilities. Evaluation, qualification, approval, and
support of PPRC configurations using channel extender products are the sole responsibility
of the channel extender vendor.
Dense Wavelength Division Multiplexor (DWDM)
Wave® Division Multiplexing (
WDM
) and Dense Wave Division Multiplexing (
DWDM
) are
used for connectivity via Metropolitan Area Networks (
MAN
), and are the basic technology of
fibre optical networking. These techniques are used for carrying many separate and
independent optical channels on a single dark fibre.
TotalStorage
Primary ESS
Secondary ESS
TotalStorage
PPRC ESCON
ESCON
director
PPRC ESCON
TotalStorage
Primary ESS
Secondary ESS
TotalStorage
PPRC ESCON
ESCON
director
ESCON
director
3 km 3 km20 km

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
105
A simple way to envision DWDM is to consider that at the primary end, multiple fibre optic
input channels, such as ESCON, Fibre Channel, FICON, or Gbit Ethernet, are combined by
the DWDM into a single fibre optic cable. Each channel is encoded as light of a different
wavelength. You might think of each individual channel as an individual color: the DWDM
system is transmitting a
rainbow
. At the receiving end, the DWDM fans out the different
optical channels. DWDM, by the very nature of its operation, provides the full bandwidth
capability of the individual channel. As the wavelength of light is, from a practical perspective,
infinitely divisible, DWDM technology is only limited by the sensitivity of its receptors, as the
total aggregate bandwidth possible.
The IBM 9729 Optical Wavelength Division Multiplexer (
MuxMaster
) enables a 50 km
distance between MuxMaster units. ESCON is used to attach the ESS to it.
The IBM 2029 Fiber Saver, also known as Dense Wavelength Division Multiplexer (DWDM),
supports ESCON, FICON, Fibre Channel, and many more protocols, enabling up to a 50 km
distance between Fiber Saver units that are ESCON attached to the primary and secondary
ESS. You can use it for Fibre Channel, network, and telephone links between the sites as
well. Figure 4-26 shows how you can reach a maximum PPRC-SYNC distance of 103 km
using the IBM 2029 Fibre Saver.
Figure 4-26 Configuration using IBM 2029 Fibre Saver
Other Channel Extension, DWDM, and Network connectivity options are:

CNT UltraNet Storage Director (USD)

CNT (INRANGE) 9801 SNS

Cisco ONS 15530/15540

Nortel Networks OPTera Metro 5200/5300
A complete and current list of PPRC supported environments, configurations, networks,
products, and required ESS LIC levels is available at:
http://www.storage.ibm.com/hardsoft/products/ess/supserver.htm
Attention: The DWDM vendor should be contacted regarding evaluation, qualification,
approval, and hardware and software support prerequisites when using their products in
an ESS PPRC configuration.
TotalStorage
Primary ESS
Secondary ESS
TotalStorage
PPRC
IBM
2029
Fiber
Saver
IBM
2029
Fiber
Saver
IBM
2029
Fiber
Saver
50 KM
50 KM
ESCON
ESCON

106
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.9.2 Configuring PPRC links
The ESCON protocol has been streamlined with less handshaking and larger frames
transmitted between ESSs. ESCON connections have to be configured between the primary
and secondary units (see 4.9.1, “ESCON links” on page 101) over which the replication will
occur.
When establishing ESCON connections between two ESS subsystems, there are some
requirements to observe:

As many ESCON links as required (depending on performance requirement and host bay
slot availability) can be used to connect the ESSs. However, only eight logical ESCON
paths may be configured between the primary and secondary LSSs.

A primary LSS can be connected via ESCON links to up to four secondary LSSs.

A secondary LSS can be connected to any number of primary LSSs, limited by the
number of ESCON links available.

PPRC links are unidirectional, because the ESCON port at the primary ESS is
reconfigured to act like an ESCON channel in a host S/390 processor. The primary
ESCON port is dedicated to PPRC.
An ESCON PPRC link can be used only to transmit data from the primary storage control
to the secondary. If you want primary and secondary volumes on each of two ESSs, you
need ESCON PPRC links in each direction. The number of links needed in each direction
depends on the total write activity to all the primary devices in each ESS.
4.9.3 PPRC paths definition
The ESCON links, as discussed above, are the physical layer of the PPRC connections. The
logical layer consists of
PPRC logical paths
, defined over the physical links. Prior to setting
up volume copy pairs, PPRC requires logical paths to be established between the primary
and the secondary ESS logical subsystem (LSS). Each LSS with primary volumes requires at
least one path to be set to the LSS that holds the secondary volumes (Figure 4-27 on
page 107).
Up to eight paths can be defined per LSS pair. For each LSS pair, each path is defined over a
different physical link. The path definition by identifying the source and the target LSSs
implicitly sets the direction of the link. A link will operate in only one direction. If you want to
have a path in the opposite direction, then you will have to use a different link. Note that if you
delete all the paths over a physical link, then you can reuse that link for paths in the opposite
direction.
When you establish a path:

You identify the source and target LSSs.

You identify the physical link that will be used between the primary and secondary LSSs.

Then, optionally, you can set the Consistency Group option (discussed in 4.5,
“Asynchronous PPRC” on page 74).
Note: In the open systems world, we refer to a logical subsystem or LSS. This equates to a
logical control unit (LCU) in the mainframe world. LSS and LCU are sometimes freely
interchanged.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
107
Each ESCON link supports 64 logical paths, so even with 16 LSS defined, you are able to set
up a logical path from each LSS to each LSS with only four ESCON PPRC links. We always
recommend that you use all available links for each LSS pair used for PPRC. That gives you
maximum protection against ESCON link failure.
For a pair of ESSs, the physical links between them can be shared with many path definitions
from different pairs of LSSs within those ESSs. In practice, many installations configure paths
to share the same links for easier management.
Figure 4-27 Logical paths
For redundancy, and for performance, it is recommended to maximize (up to eight) the
number of paths between a pair of source and target LSSs. When a pair of volumes is initially
established or at the re-synchronization time of a suspended pair of volumes, PPRC is able to
use all the defined paths in parallel. Especially with long-distance links between data centers,
you should distribute these paths evenly over so many ESCON adapters, directors, and
DWDM links as possible to keep paths functional when some of the links are temporarily
unavailable.
4.9.4 PPRC path failure alerts
When a path fails, a notification is presented in the form of an update to the WUI in the Path
Origin panel of the Paths tab or as an SNMP trap 100 to a systems management console
(SMC). The activity for that pair of LSSs continues if more paths are available, rerouting the
updated transmissions over alternate paths between that LSS pair.
Example 4-3 on page 108 illustrates the trap information for a failed path. It contains
identification of a degraded transfer rate, along with the serial numbers of the primary and
secondary ESSs involved and SAID (
PP
and
SP
) details of the path that has failed, with the
return code
(refer to Appendix D, “System Adapter ID (SAID)” on page 555 for a description).
Important: Be aware that each path definition you make will totally replace any existing
path definitions for that same pair of LSSs. So, if you are adding a path, then you should
redefine all previous path definitions between these two LSSs explicitly.
Physical
ESCON
link
Up to 64 logical paths
per ESCON adapter
port
Secondary ESS
LSS 11
:
:
Primary ESS
LSS 1F
LSS 18
LSS 13
LSS 12
LSS 10
:
LSS 10
LSS 1F
LSS 18
LSS 13
LSS 12
LSS 11

108
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Example 4-3 SNMP trap information of path failure
2002/09/05 16:03:51 CDT
PPRC Links Degraded
UNIT: Mnf Type-Mod SerialNm LS
PRI: IBM 2105-800 13-ABC12 02
SEC: IBM 2105-800 75-DEF34 02
Path: Type PP PLink SP SLink RC
1: ESCON 0001 00 0008 00 OK
2: ESCON 0020 00 0020 00 08
When the last existing path, between a pair of source and target LSSs, fails due to an error
condition, this event is considered by PPRC to be a major error condition. In an open systems
environment, an SNMP trap 101 is sent to the SMC. Under this error condition, the pairs will
be suspended and all volumes in a Consistency Group exhibit Queue Full condition (refer to
4.5, “Asynchronous PPRC” on page 74). Example 4-4 illustrates a total link-down event.
Example 4-4 SNMP trap information for total path failure
2002/09/05 16:03:42 CDT
PPRC Links Down
UNIT: Mnf Type-Mod SerialNm LS
PRI: IBM 2105-800 13-ABC12 02
SEC: IBM 2105-800 75-DEF34 02
Path: Type PP PLink SP SLink RC
1: ESCON 0001 00 0008 00 08
2: ESCON 0020 00 0020 00 08
Example 4-5 illustrates the contents of trap 102 on the resolution of the error event and the
reconstitution of the paths.
Example 4-5 SNMP trap information for path recovery
2002/09/05 16:03:58 CDT
PPRC Links Up
UNIT: Mnf Type-Mod SerialNm LS
PRI: IBM 2105-800 13-FCA36 02
SEC: IBM 2105-800 75-18592 02
Path: Type PP PLink SP SLink RC
1: ESCON 0001 00 0008 00 OK
2: ESCON 0020 00 0020 00 OK
3: ESCON 0001 00 0008 00 OK
4: ESCON 0020 00 0020 00 OK
5: ESCON 0001 00 0008 00 OK
6: ESCON 0020 00 0020 00 OK
7: ESCON 0001 00 0008 00 OK
8: ESCON 0020 00 0020 00 OK
4.9.5 Performance considerations
In the following sections, we discuss the considerations involved when setting up the Copy
Services of the Enterprise Storage Server (ESS) in order to achieve better performance. This
should help you understand the performance impact of ESS Copy Services. As there are
many different parameters that have an influence on performance, such as applications, type
of workload, and configuration of the Enterprise Storage Server, the information should serve
as a guideline when planning ESS Copy Services.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
109
As a rule of thumb, the number of ESCON paths between ESS and the type and placement of
ESCON adapters have the most significant influence on PPRC performance. Other factors
are processors, caches, and path grouping.
Keep in mind that the general ESS performance considerations, such as volume placement,
type of hard disk drives (rotational speed and number of drives per RAID array), or the
amount of storage per host adapter still apply when planning for PPRC.
Optimized PPRC communication
There were certain modifications made to the ESCON protocol used for PPRC
communication of the Enterprise Storage Server, in particular:

A larger frame size, which results in less overhead during PPRC communication.

Less handshaking between the two communicating ESSs, which makes transfer of data
more efficient. The handshake was reduced from six down to three exchanges.
Cache and NVS sizes, number of processors
One of the major considerations about capacity planning for PPRC used to be the cache and
NVS sizes of both application and secondary ESSs. Today, there are multiple cache sizes
available, depending on the model of the ESS configurations. So you will need to take these
into account when planning a PPRC configuration. PPRC requires extra I/O activity to the
back end of the ESS as compared to non-PPRC boxes, so it is important to consider larger
cache sizes, or the cache backstore ratio when using PPRC.
With the ESS F20, there were diminishing returns from using more than eight PPRC links in
terms of 4 KB operations. For the ESS 800 with turbo option (two additional and faster
processors in each ESS cluster), the per link throughput scales well to 16 links. In
environments with a very high number of I/O operations per second, the ESS 800 turbo
model provides better PPRC performance and keeps the I/O response time stable in a wider
range of workloads than the ESS 800 standard model. This effect increases with the number
of ESCON links. Therefore, when a large number of links is used, the ESS 800 with turbo
option is recommended for the best possible performance.
You can call your IBM representative to help you determine the resources that are needed to
implement PPRC, and the impact of PPRC on the application system. There are some
modeling tools, like DiskMagic, that can predict the PPRC effect on the application system
before the real implementation takes place.
Number of ESCON paths between ESSs
The number of ESCON channels needed between PPRC primary and secondary LSS
depends on the response time requirements of the application servers in a synchronous
environment and the available bandwidth between primary and secondary ESS subsystems
in an asynchronous environment.
Always make sure that you are using an appropriate number of physical ESCON links for
PPRC between the primary and the secondary ESS. Increasing the number of the physical
ESCON links will increase the maximum overall bandwidth for updating the secondary
volumes. Using multiple physical ESCON links for a PPRC pair (maximum of eight logical
paths per LSS) will improve the response time of an I/O request from the host and minimize
the out-of-sync tracks in a PPRC-XD environment. Keep in mind that too few physical links
may result in a bottleneck. A minimum of four links between the primary and secondary ESS
are recommended.

110
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Type and placement of the ESCON adapters used for PPRC
Until 2002, the ESCON host adapters on the ESS used a 32-bit wide bus. With the
introduction of 64-bit ESCON Host Adapters for the ESS 800, ESCON speed, efficiency, and
throughput are improved. Use of 64-bit ESCON Host Adapters matches the 64-bit internal
buses within the ESS 800 and therefore the hardware capability may be fully exploited. The
64-bit adapter has about 10% less overhead than the 32-bit adapters used previously,
resulting in lower response times and additional throughput. The throughput for PPRC
establish will improve roughly 10% compared to the 32-bit ESCON adapters.
All models of the ESS 800 now come standard with 64-bit ESCON adapters. If you have older
ESSs with 32-bit adapters, then you might consider upgrading to 64-bit adapters.
Configurations with 64-bit ESCON adapters on one end of a PPRC link and old 32-bit
adapters at the other end work also. As might be expected, performance will be somewhere
in between what would be seen for a link using two 64-bit adapters and a link using two 32-bit
adapters.
Distribute the ESCON adapters used for PPRC and PPRC-XD evenly across the two clusters
and the host adapter bays of the ESS. This will distribute the PPRC workload over multiple
buses and both clusters.
For example, if there are four ESCON adapters used for PPRC between two Enterprise
Storage Servers, place one ESCON adapter in each of the host adapter bays.
Grouping of physical and logical paths
A physical path describes the physical ESCON connection between two Enterprise Storage
Servers. A logical path is the connection used for the PPRC copy pair, either between two
volumes or two logical subsystems. There could be multiple logical connections established
over a single physical connection. This will most likely be the case in a real environment.
Also, consider that multiple logical paths using the same physical ESCON links(s) will share
the bandwidth of these ESCON links(s). If there are performance-critical PPRC pairs, we
recommend that you separate them on dedicated physical paths so that the I/O traffic of the
data copy from the primary to the secondary side will not interfere with I/O traffic of less
critical PPRC pairs.
When you plan to use Synchronous PPRC as well as PPRC Extended Distance between a
pair of ESSs, IBM recommends that you establish separate logical paths over separate
physical paths for the copy pairs managed by each of the modes. In other words, for your
Synchronous PPRC copy pairs, use one set of logical and physical paths between the source
and target LSSs; for your PPRC Extended Distance copy pairs, use another set of logical and
physical paths between the source and target ESSs.
By keeping the paths separate for the two copy modes, the updates to PPRC Extended
Distance target volumes minimize the effect on the I/O performance of the Synchronous
PPRC pairs. This recommendation only applies to environments where the distance between
primary and secondary ESSs does not exceed the synchronous range. This recommendation
does not apply to the following situations:

Using the PPRC Extended Distance mode to establish copy pairs that you convert to
synchronous pairs after the bulk copy is complete.

Converting PPRC Extended Distance copy pairs to Synchronous PPRC copy pairs on an
LSS-to-LSS level.

Re-establishing PPRC Extended Distance pairs that were previously Synchronous PPRC
copy pairs on an LSS-to-LSS level.

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
111
When using Asynchronous Cascading PPRC, different physical ports should be used on the
intermediate ESS for the primary–intermediate and the intermediate–remote connections.
You should keep the distance between the local and intermediate sites as close as possible to
minimize the performance impact of the Synchronous PPRC operation.
Setup of the secondary ESS
For disaster recovery reasons, you may be doing PPRC between two or more different
Enterprise Storage Servers. Under normal operating conditions, you always have a source
(primary side) and a target (secondary side) of a PPRC pair.
One single ESS could have up to four secondary ESSs. However, the number of primary
servers of a single secondary server is only limited by the number of available ECSON links.
So it may be the case that different primary storage servers are connected to the same
secondary ESS. In that case, the I/O traffic of multiple primaries has to handled by a single
secondary ESS.
Furthermore, it may be possible that secondary volumes from different primary storage
servers are placed on the same disks within the same array (rank). In that case, the response
time of each primary storage server will increase if other primaries are doing I/O at the same
time, as all requests are handled simultaneously.
Therefore, when planning your Enterprise Storage Server network, keep in mind how many
primary storage servers are connected to the same secondary. Distribute the I/O load evenly
across the secondary storage server.
Try to distribute the secondary volumes belonging to PPRC pairs from different primaries
across all available RAID arrays within the secondary ESS.
4.10 PPRC using Fibre Channel links
Fibre Channel can be used as the communication link for PPRC between primary ESS and
secondary ESSs. This allows exploitation of existing Fibre Channel infrastructure and
capacity.
PPRC over Fibre Channel requires PPRC Version 2 (feature #85xx) and is supported on the
ESS Model 800 or ESS Model 750 at LIC level 2.3.x with Fibre Channel/FICON host adapters
(features #3024 and #3025).
When compared with ESCON links, Fibre Channel reduces the link infrastructure by at least 4
to 1 with equivalent or better performance.
When implemented over Fibre Channel links, PPRC can be managed using:

ESS Copy Services Web User Interface, ESS API, and ESS Copy Services CLI, for all
environments

ANTRQST API, TSO commands, and ICKDSF, for z/OS environments

ICKDSF, for z/VM® and VSE/ESA™ environments
Note: The present section addresses the specifics for Fibre Channel connectivity. This
information should be complemented with the general discussion on PPRC paths and links
presented in 4.9, “PPRC connectivity” on page 101.

112
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.10.1 Configuration guidelines
An ESS Model 800 or 750 Fibre Channel port can simultaneously be:

Sender for a PPRC primary

Receiver for a PPRC secondary

Target for FCP host(s) I/O from Open systems and Linux on zSeries
A PPRC Fibre Channel link is full duplex. This means that one link can have a PPRC path
established in one direction and simultaneously have a PPRC path established in the
opposite direction. This contrasts with ESCON PPRC links, which are uni-directional. ESCON
ports are dedicated and so cannot be used for host I/O.
As for ESCON PPRC links, PPRC Fibre Channel links support both Synchronous PPRC and
PPRC extended distance (XD).
Although one PPRC Fibre Channel link would have sufficient bandwidth for most
environments, we recommend configuring two PPRC Fibre Channel links between each
primary and secondary ESS, as shown in Figure 4-28.
Figure 4-28 Configuring for availability
PPRC Fibre Channel links can be direct-connected (dedicated) or switches used (up to two
switches).
Figure 4-29 on page 113 shows direct PPRC Fibre Channel links. The PPRC Fibre Channel
links are direct connected between two Fibre Channel ports. There is no other sharing of
these Fibre Channel ports; they are used exclusively for PPRC. Both ports can be sending
and receiving at the same time due to the full duplex capability of Fibre Channel. Dedicated
Fibre Channel ports guarantee no interference from host I/O. This is recommended, as
Synchronous PPRC is time critical and should not be impacted by host I/O activity. The
PPRC ports can be used to provide connectivity for all LSSs within the ESS and can contain
multiple PPRC paths.
Configure for availability
Adapter
FCP
Adapter
FCP
Adapter
FCP
Adapter
FCP
HA Bay 2
Adapter
FCP
Adapter
FCP
Adapter
FCP
Adapter
FCP
HA Bay 2
Adapter
FCP
Adapter
FCP
Adapter
FCP
Adapter
FCP
HA Bay 1
Adapter
FCP
Adapter
FCP
Adapter
FCP
Adapter
FCP
HA Bay 1
PPRC links
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Port
Primary ESS
Secondary ESS

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
113
Figure 4-29 Direct PPRC FCP links
Figure 4-30 shows a typical configuration for an open systems environment using two
switches. As in the direct-connected configuration, we recommend that the PPRC Fibre
Channel ports be dedicated and not used for host I/O.
Figure 4-30 Typical open systems host and PPRC connectivity using Fibre Channel
Logical paths
A logical path is a logical (zoned) connection between the ESS Fibre Channel Host Adapter
and the server I/O Fibre Channel Adapter card. This constitutes the medium for host to ESS
disk I/O. This differs from a physical connection, which is the collection of elements (links,
ports, adapter cards, and switch ports) that provide the physical path for the information flow.
Figure 4-31 on page 114 shows how multiple logical paths can use the same physical path.
Direct PPRC FCP links
bi-directional / full duplex
Adapter
Port
FCP
Adapter
Port
FCP
SAN Fabric
Adapter
Port
FCP
Adapter
Port
FCP
Dedicated
Dedicated
Multiple PPRC paths
can
share the same port
Direct connect point-to-point
PPRC Secondary
and / or
PPRC Primary
PPRC Primary
and / or
PPRC Secondary
PPRC
Adapter
Port
FCP
Adapter
Port
Adapter
Port
Adapter
Port
FCP
Dedicated
Dedicated
Server 1
Server 2
FCP
FCP links
PPRC
FCP links
FCP
FCP
Dedicated PPRC FCP ports
Shared host FCP ports
PPRC Secondary
and / or
PPRC Primary
PPRC Primary
and / or
PPRC Secondary

114
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 4-31 Logical paths between servers and LSSs
Logical paths also apply to PPRC connections. A PPRC logical path is a logical connection
between the sending LSS and receiving LSS. A PPRC link can accommodate multiple PPRC
logical paths.
Figure 4-32 shows that if we have a 1:1 ESS mapping, we would have three logical paths over
the PPRC link: LLS1 to LSS1, LLS2 to LSS2, and LSS3 to LSS3. If the volumes in LSS1
mapped to all three secondary LSSs and similarly for LLS2 and LLS3, there would be nine
logical paths over the PPRC link. Note that we recommend a 1:1 LSS mapping.
Figure 4-32 Logical paths for PPRC
PPRC Fibre Channel has not changed certain PPRC architectural characteristics:

A primary LSS can maintain paths to a maximum of four secondary LSSs. Each
secondary LSS can reside in a separate ESS.

Up to eight logical paths per LSS-LSS relationship can be defined. Each PPRC path
requires a separate physical PPRC link. This is illustrated in Figure 4-33 on page 115.

The ESS Copy Services Server (CSS) can manage up to 4096 volumes/LUNs (2048
pairs). This is the total number, including primary and secondary volumes/LUNs, as well
as source and target FlashCopy volumes/LUNs.
Open system servers
Fully shared configuration
ESS
Port
2 logical paths
LSS 2
LSS 3
LSS 1
2 logical paths
Server 1
Server 2
adapter
p1
switch
adapter
adapter
p2
p3
ESS 1
Disk Storage Server (DSS)
Port
1 logical path
LSS 2
LSS 3
LSS 1
1 logical path
1 logical path
Port
1 logical path
LSS 2
LSS 3
LSS 1
1 logical path
1 logical path
3-9 logical paths
1 Link
PPRC paths
ESS 2
switch

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
115
Figure 4-33 Up to eight paths per LSS-LSS relationship
Some PPRC implementation constraints have been relieved when using Fibre Channel links.
The new limitations are illustrated in Figure 4-34.

An FCP port can host up to 1024 logical paths. These are the logical and directional paths
that are made from LSS to LSS. The ESCON limit was 64.

An FCP path (the physical path from one port (SAID) to another port (SAID)) can host up
to 128 logical paths (PPRC paths). The ESCON limit was 64.

An FCP port can accommodate up to 126 different physical paths (ESS port (SAID) to
ESS port (SAID) through the SAN).
Figure 4-34 Logical path/physical path limits
Table 4-6 on page 116 compares FCP versus ESCON architectural characteristics.
8 Links
Secondary
LSS 2
Primary
Each link
hosts 2 PPRC
paths
LSS 1
LSS 2
LSS 1
8 PPRC paths
8 PPRC paths
8 PPRC paths
8 PPRC paths
1024 logical paths
Port
LSS 2
LSS 1
LSS 16
LSS ....
Port
128 logical paths
126 physical paths
Port
Port
ESS2
ESS1
PPRC
LSS 2
LSS 1
LSS 16
LSS ....
Host1
Host2

116
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Table 4-6 Fibre Channel versus ESCON characteristics
4.10.2 Distance considerations
The distance between Fibre Channel ports without any extending infrastructure is dependent
on the Fibre Channel port type. The ESS uses 2 Gbit GBICs on its Fibre Channel ports. For
short wave GBICs using 50 micron multimode fiber, the maximum distance is 200 meters.
With 62.5 micron multimode fiber, the maximum distance is 250 meters. For long wave
GBICs over single-mode fiber, the maximum distance is 10 km.
Figure 4-35 illustrates the Fibre Channel maximum distances when no networking devices
are used to extend those distances.
Figure 4-35 Fibre Channel distances without extending devices
FCP compared to ESCON FCP ESCON
PPRC primary port accepts host I/O.Yes No
PPRC secondary port accepts host I/O.Yes Yes
Link is full duplex (have paths established in both directions
simultaneously).
Yes No
Support Synchronous PPRC.Yes Yes
Supports PPRC Extended Distance.Yes Yes
Number of secondary LSSs per primary LSS.4 4
Number of PPRC logical paths per LSS.8 8
Number of logical paths from a primary ESS.1024 1024
Number of logical paths per port.1024 64
Number of logical paths per physical path.128 64
Number of physical paths per port.126 64
Adapter
Port
Adapter
Port
Adapter
Port
Adapter
Port
PPRC Secondary
and / or
PPRC Primary
PPRC Primary
and / or
PPRC Secondary
2 Gbit
2 Gbit
2 Gbit
2 Gbit
short waveshort wave
wave
long
wave
long
10,000 meters
250 - 200 meters
No extending devices inbetween

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
117
4.10.3 SAN fabric and networking
In addition to Fibre Channel link direct connection, PPRC over Fibre Channel is also
supported with the use of SAN fabric products and DWDMs. All Fibre Channel switches and
directors supported by the ESS today are also supported for PPRC over Fibre Channel.
For extending the PPRC Fibre Channel distances, the following DWDMs are supported:

ADVA FSP 2000

CIENA CN 2000 Storage Extension Platform

Cisco ONS 15530/15540

CNT Edge Storage Router

Nortel Networks OPTera Metro 5200/5300
A current list of supported environments, configurations, networks, and products is available
at:
http://www.storage.ibm.com/hardsoft/products/ess/supserver.htm
4.10.4 WWNN and WWPN
The WWNN (World Wide Node Name) is a single name given to an entity that is in a Fibre
Channel network. A WWPN (World Wide Port Name) is the name of one of the ports into that
entity. If, for example, you look at a Windows server with multiple HBAs, the code in the first
adapter to get enabled will assign the WWNN of the server, and the second, third, and fourth
adapters will all 'inherit' the WWNN but will represent themselves as the WWPN that is
burned into the HBA.
In the case of ESS, there is a single WWNN of the ESS that is reported in the ESS Specialist
Welcome panel (see Figure 4-36).
Figure 4-36 ESS WWNN information in the ESS Specialist Welcome panel
WWNN

118
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
If you go into the ESS Specialist and click on each of the Fibre Channel host adapters (see
Figure 4-37), you will see that they all have a unique WWPN, but that the last four or six digits
are all the same. In this case, instead of the burned-in WWPN, the ESS dynamically assigns
the WWPN for each adapter. This way, if there is a need to replace a host adapter, then it will
retain its previous identity in the Fibre network.
Figure 4-37 WWPN of ESS Fibre Channel Port 4-4-A
4.10.5 Recognizing the ESS ports within the SAN fabric
In the ESS, there is a function called "locally administered WWPNs". The WWPN (World
WIde Port Name) of a Fibre Channel port is calculated by a byte-level operation from the ESS
node WWNN (World Wide Node Name, the one shown by the entry panel of ESS Specialist)
and the adapter's port location in the ESS bay. Thus, the adapter's WWPN remains
unchanged when the adapter is replaced.
A nice consequence of this characteristic is that you can easily recognize your ESS ports in
the SAN: all Fibre Channel switch GUIs have some windows displaying the device’s WWPNs
attached to the fabric ports (with Brocade switches, it is the Name Server panel). You see
something there similar to "IBM 2105 ... 50:05:07:63:00:c1:84:c8 ...". When you know how the
WWPNs are identified in the ESS, you then can verify that the ESS ports are attached to the
fabric ports as you want. This knowledge will also help you in other fabric management
activities, for example, verifying that the correct WWPNs appear in zoning lists.
Note: When looking at the Information Panel for a selected Fibre Channel port from the
ESS Copy Services Web User Interface Paths panel (see 6.7.5, “Path Information Panel
button” on page 217), you will see the WWNN and WWPN of the Fibre Channel switch to
which the ESS FCP port is connected to.
WWPN

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
119
The way to determine how the FCP ports are assigned their WWPN in the ESS is as follows:
take the ESS node WWNN (you can get if from the ESS Specialist Welcome panel, as shown
in Figure 4-36 on page 117). Byte 6 in the WWNN is always c0 (in hexadecimal code). You
get an FCP adapter's WWPN by replacing the x’c0’ with the location code (c1...d0) for that
adapter as per the diagram and example illustrated in Figure 4-38.
Figure 4-38 Determining the WWPN of an FCP port in the ESS
4.11 Asynchronous PPRC Bandwidth Estimator
The Asynchronous PPRC Bandwidth Estimator is used by IBM ITS to support clients in the
planning and sizing needed for Asynchronous PPRC. The customer requirements have to be
analyzed to obtain the input data for the tool. This input data would include the required data
currency, the distance to the remote location, the number of ESS and HBAs, the
compressibility of the data and the current workload characteristics of all the Open Systems
and z/OS data that should be part of the Asynchronous PPRC session.
The tool calculates the expected bandwidth required for the solution and provides a
recommendation for the number of required remote links for several available link types, such
as OC-3, OC-12 T3, and many others.
Please contact your IBM representative for this bandwidth analysis.
4.12 Practical examples of PPRC
This section outlines some of the practical uses of PPRC in both modes of operation,
synchronous and extended distance.
4.12.1 Site migration
Site migration without remote copying facilities must be performed often using backups onto
tape volumes and then restoring them onto a remote disk subsystem. Incremental backups
need to be performed and applied regularly to the remote site. This method provides a
point-in-time like recovery scenario. When the time comes to do the actual site migration, a

120
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
final backup and apply would need to be performed to get an up-to-date copy of the
application data. The time elapsed for migration would be entirely dependent and elongated
by the backup/apply process.
Site migrations can now be done with PPRC in a much shorter time with the minimum
possible disruption to the production process. The application site volumes to be migrated
can be receiving updates from the application, while keeping a copy at the remote site. For
asynchronous mirroring using PPRC-XD, an appropriate application checkpoint needs to be
defined to allow application writes to be quiesced for a very short time, to let the secondaries
catch up.
Once the decision has been made to swap sites, all that is needed is to quiesce the
application writes and allow them to propagate to the remote site. Once all the volumes are in
duplex, the relationship can be terminated, and the application can be started again on the
remote site. The amount of application downtime for this procedure depends on the
availability of the application servers on the remote site.
Figure 4-39 Site migration
4.12.2 Synchronous PPRC using static volumes
Remote data migration can be achieved with Synchronous PPRC using static volumes. A
static volume is a volume that is not receiving any write updates. This particular
implementation allows the use of Synchronous PPRC over long distances, beyond the
supported 103 km (300 km with PPRC over Fibre Channel), while taking advantage of the
excellent throughput of PPRC when it does the initial copy or the re-synchronization of
volume pairs (duplex pending state).
This powerful way of PPRC data copying, combined with the latest microcode improvements
in the supported channel extenders, makes the static volume implementation ideal for data
migration, data copying, or remote backup over long distances.
Start Up PPRC-SYNC/XD
Establish all volumes
At beginning of data center cutover window
Quiesce applications
Allow PPRC-XD to catch up (time would be short)
Once PPRC-XD caught up, data is ready at remote location
Significant
reduction in
work and effort
required to migrate
data centers over
long distance
B
L
X
C
M
Y
ESS 2:
Mirrored
Data
Production
Servers
Backup /
Admin
Servers
ESS 1:
Production
Data

Chapter 4. Peer-to-Peer Remote Copy (PPRC)
121
This PPRC implementation can also be used for disaster recovery solutions over long
distances when:

The applications in consideration can be quiesced.

Solutions are based on switching database log files, and transmitting when inactive.
Additional information on this particular implementation can be found in the redbook IBM
TotalStorage Enterprise Sorage Server PPRC Extended Distance, SG24-6568.
4.12.3 Database log transmission
Database recovery solutions based on switching active database logs and transmitting the
then inactive log and the bootstrap data set (BSDS) can be efficiently implemented with
PPRC. Once at the remote site, the log is applied onto the shadow database (see
Figure 4-40).
Figure 4-40 Database log transmission
4.12.4 Off-site backups
Using PPRC to perform off-site backups, there is no need for transporting media to the vault
site. Off-site backups can be performed using split-mirror implementations using PPRC with
FlashCopy (see Figure 4-40).
#2: For PPRC-XD wait
for catch-up
PPRC
Replication
ESS 2:
Mirrored
Data
Production
Servers
Backup /
Admin
Servers
ESS 1:
Production
Data
log1
log2
log3
log1'
log2'
log3'
D
#3: Shadow process
applies
log1'
to
shadow database
#1: Application
dynamically
switches
from
log1 to log2

122
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 4-41 Split mirror implementation
1.Establish volume pair A-B in PPRC-XD relationship
2.Run in duplex/duplex-pending XD mode until point-in-time
consistent copy needed
3.When consistent copy needed, quiesce the application
4.If PPRC-XD, wait for catch up and then freeze (or suspend)
5.With the pairs suspended, the application can resume
6.FlashCopy secondary B to tertiary C (no impact to host)
7.When FlashCopy is initiated, resume mirroring SYNC/XD
8.Go back to Step 2.
SECONDARY PRIMARY
PPRC-SYNC/XD
CHAN EXT
CHAN EXT
Vol B
FlashCopy
Vol C
Vol A
Sequence of operation

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
123
Chapter 5.
ESS Copy Services Web User
Interface prior to LIC 2.2.0
There are three different methods of using the ESS Copy Services in the open systems
environment:

A Web-based Interface

A Java-based Command Line Interface (CLI)

And, starting with LIC level 2.3.0, the ESS API has been enhanced to support Copy
Services configuration and use for PPRC and FlashCopy
Very important to note is the fact that the Web-based user interface panels and icons have
changed with LIC 2.2.0 first and later when LIC 2.3.0 became available.
In this chapter we explain how to use and set up the ESS Copy Services Web User Interface
(WUI) when the ESS is at a LIC level prior to 2.2.0. The descriptions and panels correspond
to what users will have if their LIC level is below 2.2.0.
The Web User Interface for ESSs with LIC level 2.3.0 is discussed in Chapter 6, “ESS Copy
Services Web User Interface: LIC 2.2.0 and later” on page 189. The usage of the Command
Line Interface is described in Chapter 7, “ESS Command Line Interface” on page 377. The
ESS API is discussed in Appendix A, “ESS Application Programming Interface (API)” on
page 533.
5

124
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.1 LIC level considerations within this chapter
All the procedures, screens and panels, PPRC options, and FlashCopy options described in
this chapter refer to the LIC level Version prior to 2.2.0.
When you upgrade the ESS LIC level to 2.2.0 or higher:

The ESS Copy Services WUI volume panel presentation will change.

The definition of the Copy Services Domain will change.

The FlashCopy will not change until you upgrade the FlashCopy feature to Version 2.

The PPRC options will not change until you upgrade the PPRC feature to Version 2.
You may refer to Chapter 6, “ESS Copy Services Web User Interface: LIC 2.2.0 and later” on
page 189 to get more information about the new volume panel presentation and the new
FlashCopy and PPRC options.
5.2 Overview and requirements
ESS Copy Services runs within the Enterprise Storage Server. When the ESS is running LIC
prior to 2.2.0, one of the ESS clusters in the Copy Services Domain has to be defined as the
Primary Copy Services server and is responsible for maintaining all Copy Services related
information. Optionally, you can define a second ESS cluster in the Copy Services Domain to
act as the Backup Copy Services server to provide availability of service. On each ESS
cluster that is intended to use Copy Services, there is a Copy Services client running that
communicates with the Active Copy Services server. For further information, refer to 2.1,
“Copy Services terminology” on page 18.
Access to Copy Services is provided through a Web browser. Using a Web browser offers the
possibility to easily control the ESS copy functionality over the network from any platform for
which the browser is supported.
A Web server running in your ESS provides a Web interface that you can use to manage the
ESS through a Web browser. The Web browser must be running on a workstation connected
to the ESS through the IBM TotalStorage Enterprise Storage Server Network (
ESSNet
). The
ESSNet is the ESS access facility that IBM installs when they install your ESS. The ESSNet
consists of a switch (hub) and other networking components and the dedicated IBM
TotalStorage Enterprise Storage Server Master Console (
ESS Master Console
).
The

ESS Master Console replaces the

ESSNet Console, the workstation included in some
earlier versions of the ESSNet

package. For additional information on the ESS Master
Console, see the IBM TotalStorage Enterprise Storage Server User’s Guide, SC26-7445, on
the ESS Web site. For additional information on the ESSNet, see the IBM TotalStorage
Enterprise Storage Server Introduction and Planning Guide, GC26-7444, on the ESS Web
site.
In addition to using a Web browser on the

ESSNet Master Console to connect to your ESS,
you can also use a Web browser running on your own workstation, either by connecting your
workstation directly to the ESSNet hub, or by connecting your workstation to your intranet and
connecting your intranet to the ESSNet hub. If you use your own workstation, IBM
recommends that it have at least 128 MB of memory.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
125
The ESS Copy Services require one of the following Internet browsers:

Netscape Communicator

Microsoft® Internet Explorer (MSIE)
For supported versions of Netscape Navigator and Internet Explorer, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.
5.3 Using a browser to access the ESS
You can either use the ESS Master Console or a Web browser installed on a workstation
connected through the ESSNet to access the ESS.
The ESS Master Console differs from other workstations in the way you access your ESS.
The desktop of the ESS Master Console has a Netscape icon with the label ESS Specialist
Launcher. That icon accesses a panel, which provides a list of IP addresses of your ESS
clusters. Clicking on the address of the cluster that you wish to access will cause the ESS
Launch panel resident in the selected cluster to open.
If you use a workstation other than the ESS Master Console to access your ESS, you must
enter the desired cluster address in the Address field of your Web browser. The address that
you click in the
Specialist Launcher
of your ESS Master Console or that you type in the
Address field of your Web browser is the host name alias or the dotted decimal IP address of
one of the clusters of your ESS. IBM configures these addresses in your ESS at installation
time, based on the addresses you enter in the Communications Resources work sheet that
you submit for your ESS. For details, see the IBM TotalStorage Enterprise Storage Server
Introduction and Planning Guide, GC26-7444.
The first page to open, the ESS Launch

panel, is the same for either of the two clusters in the
ESS, so access to either cluster allows you to configure the entire machine. The ESS Launch
panel contains buttons that you click to access ESS Copy Services (Figure 5-1 on page 126).
This connects the browser to the ESS that is specified as the Copy Services server. If you
have not previously selected the Copy Services

button, you will be prompted for the user
name and the password before starting the Copy Services Web User Interface.

126
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-1 ESS launch panel
The message panel shown in Figure 5-2 will be displayed while connecting to the Copy
Services server.
Figure 5-2 Copy Services start message
Once the connection to the Copy Services server is successful, the main menu of the
Copy
Services Web User Interface
will be displayed (Figure 5-3 on page 127). From here, you can
access all Copy Services menus by selecting one of the buttons on the left side.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
127
Figure 5-3 Main menu of ESS Copy Services Web User Interface
5.3.1 Failure to connect to ESS Copy Services
If you receive the following message when you click Copy Services in the navigation frame of
the ESS Launch page:
Failed to connect to Copy Services Server. Server may be down or not configured and cannot
access ESS Copy Services
the failure is most likely the result of at least one of the following three conditions:

The Primary Copy Services server has not been defined. If you have successfully
accessed ESS Copy Services before, this would not be the source of the problem. If this is
the first time that you are connecting to ESS Copy Services, see 5.3.2, “Location of ESS
Copy Services server” on page 128.

The primary Copy Services server is not running. See 5.3.3, “Restarting ESS Copy
Services” on page 128 for instructions.

Your Web browser has a temporary operational or connection problem.
For Web browser problems, take these corrective actions:
1.Close any open ESS Copy Services browser panel.
2.Access ESS Copy Services again by clicking Copy Services.
3.If step 2 is unsuccessful, close all browser panels (if you have ESS Specialist running, first
complete any unfinished task definition), and then restart your browser.
Note: For all three conditions noted above, if you have the CLI active on one of your host
systems, you can verify that there is a connection to the active Copy Services server with
the
rsTestConnection
command. For details, see 7.1.4, “Copy Services CLI Command
description” on page 380.

128
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.Again, access the ESS Launch page and click Copy Services.
5.If Copy Services still does not launch, consider restarting the workstation that is running
your browser.
5.3.2 Location of ESS Copy Services server
You can verify that the Copy Services server is defined by using the Tools panel. From the
ESS Launch panel, click Tools. The Tools panel displays the topic “Tools: Copy Services
Trouble Shooting and Disaster Recovery”. To determine which Copy Services server is the
active server, click the corresponding topic. Click Find the active Copy Services server. The
Current Configuration Table is displayed (Figure 5-4).
Figure 5-4 Configuration table of ESS Copy Services
Verify that there is an IP address displayed for Currently Configured Primary Server. If there is
an IP address, the primary Copy Services server is defined. If not, the primary Copy Services
server has not been defined.
5.3.3 Restarting ESS Copy Services
If ESS Copy Services is not available, you might want to restart ESS Copy Services. Be
aware that you need to have administration authority to do this.
To restart Copy Services, click the Tools button (as in the previously described procedure)
and the Tools panel displays. On that panel, review the message
Recovering from a Failure
to Connect to the Copy Services Server
to link to the restart procedure. Be sure you
understand the result of implementing a restart before proceeding. When you use this
function, be aware that:

You will lose any PPRC or FlashCopy tasks that have not completed.

Established PPRC and FlashCopy relationships are maintained.

You cannot submit any additional tasks through the CLI until ESS Copy Services has
reinitialized.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
129

You must exit the Copy Services panel and re-enter it to receive valid updates from the
Copy Services server.
If you decide to restart ESS Copy Services, click Reset ESS Copy Services. The message
Resetting the ESS Copy Services
appears.
In the Available Actions section (Figure 5-4 on page 128), you can click Reset to Primary to
restart Copy Services with the primary Copy Services server as the active Copy Services
server.
Alternatively, you can click Reset to Backup to restart Copy Services with the Backup Copy
Services server as the active Copy Services server. (For an explanation of these terms, see
2.1, “Copy Services terminology” on page 18.).
5.3.4 How to switch to the Backup server
To test your disaster recovery process, you can create a simulation of a disaster at your
production site and restart the Copy Services server at the backup site.
Let us assume that you have two ESSs: ESS1 and ESS2. The primary Copy Services server
is defined on ESS1, and the Backup Copy Services server is defined on ESS2. Perform the
following steps to transfer control of Copy Services from the primary to the Backup server:
1.On the ESS launch panel, click Tools. The Tools panel opens, as shown in Figure 5-5.
Figure 5-5 The Tools panel
Attention: Use this function only if you are certain that no other recovery options are
available.
Note: You would take this action if the primary Copy Services server was unavailable
because of a disaster or emergency situation. For more information, see 9.4, “Copy
Services server failover” on page 461.

130
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.On the Tools panel, click Reset do backup.
3.Within the Reset to backup section, click Reset ESS Copy Services and the
Configuration Table of ESS Copy Services displays (Figure 5-6). This will show the
Primary Copy Services server, the Backup Copy Services server, and the current Active
Copy Services server.
Figure 5-6 Configuration table of ESS Copy Services
4.Click Disable to deactivate the current Active Copy Services server.
5.Click Reset to Backup to make the Backup Copy Services server at the recovery site the
Active Copy Services server.
5.3.5 Steps following a disaster at the production site
If a disaster occurs at the production site and your Primary Copy Services server is running
on this site, you must invoke ESS at the recovery site as the active Copy Services server.
For the following procedure to work, you must have created the proper recovery tasks (see
9.4, “Copy Services server failover” on page 461).
To switch to the recovery site, establish a Web browser connection to the ESS at the recovery
site and perform the steps in 5.3.4, “How to switch to the Backup server” on page 129 to
enable the Backup server as the active Copy Services server.
Open the Copy Services Web User Interface and terminate all PPRC pairs across all ESSs.
Direct the commands to the secondary volumes, because the primary LSSs may not be
accessible. See 5.11.8, “Terminating a PPRC copy pair” on page 177 for instructions on
terminating pairs.
Attention: You can create and save tasks when the active server is the Backup server, but
these tasks are lost when you perform the Reset-to-Primary function, which re-establishes
the primary server as the active server.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
131
Perform any necessary host specific tasks required to revise the operating system
configuration with the data held on the secondary volumes. Then bring up the applications on
the disaster servers.
Once the production site has been restored, establish PPRC paths from the recovery site to
the production site and copy the entire contents of a PPRC volumes at the recovery site to the
volumes at the production site.
Once all the PPRC volumes are in duplex state and you have decided to switch back to the
production site, perform this procedure in the reverse direction.
5.4 Volumes panel
Volumes are defined with the ESS Specialist in order to provide a fixed storage capacity to the
connected host system. They are the base components of each data copy task. The ESS
assigns each volume a unique eight digit identifier (ID). This identifier is used to address each
volume within the ESS.
From the Volumes menu, you will be able to:

Establish and withdraw FlashCopy pairs.

Establish, suspend, and terminate Synchronous PPRC copy pairs.

Establish, suspend, and terminate PPRC Extended Distance copy pairs.

Convert a PPRC Extended Distance copy pair to Synchronous PPRC.

Establish multiple volume-pair relationships for FlashCopy and PPRC.

Find a volume.

Display volumes based on a filter.

View information about a volume.
Figure 5-7 shows the entry panel of the Volumes panel.
Figure 5-7 Volumes panel

132
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The source and target logical subsystem are specified in the following way: Device type (4
digits):ESS Serial number (5 digits):LSS number (2 digits). An example would be a logical
subsystem that is addressed by 2105:2259:16.
The Volumes menu shows all volumes defined within the LSS. Below each volume you will
find its unique serial number. The volume icon indicates if it is used in any copy relationship
(source or target), or if it is not part of a copy pair at all.
Table 5-1 explains the meanings of the volume icons that appear on the Volumes panel.
Table 5-1 Meaning of icons on Volume panel
Volume icon Appearance Meaning
Gray solid (simplex volume) The volume is in simplex state, meaning that
it is not in a copy pair created by FlashCopy
or PPRC
Gray over blue
Primary (source) volume
Synchronous PPRC, initial copy in progress
(copying entire volume).
Gray over red
secondary (target) volume
Blue solid
Primary (source) volume
Synchronous PPRC volume in a
non-suspended state.
Red solid
Secondary (target) volume
Blue wedge in gray cylinder
Primary (source) volume
PPRC Extended Distance volume in a
non-suspended state.
Red wedge in a gray cylinder
Secondary (target) volume

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
133
5.4.1 Volume Information Panel button
You can get more detailed information about a single volume by selecting the volume and
clicking the Information Panel button, as shown in Figure 5-8.
Figure 5-8 Volume Information Panel
Blue and white stripes
Primary (source) volume
PPRC (synchronous or Extended Distance)
in a suspended state.
Red and white stripes
Secondary (target) volume
Blue lightning bolt on gray
cylinder
Source volume
FlashCopy in progress. The FlashCopy icon
on the target (the small white square)
indicates that this is a duplicated volume.
Red lightning bolt on gray cylinder
Target volume
White square on gray cylinder
I-am-a-target Icon
(FlashCopy indicator)
After a successful FlashCopy, the FlashCopy
icon remains until you issue a Withdraw
FlashCopy command to the target volume
(see 5.10.3, “Withdrawing a FlashCopy pair”
on page 155) or until you use the volume as a
source.
Volume icon Appearance Meaning

134
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The following information is displayed for a volume:

The storage server (ESS) and logical subsystem (LSS) designations

The volume number

The volume serial number

The LSS type

The volume type (fixed block for open system; 9337, 3390, and other types for S/390)

The volume capacity

Peer status (indicates whether the volume is a source or target volume and the copy
mode)

Any active FlashCopy operations

Whether Read from Secondary is enabled

Information about a companion volume if a copy pair exists
5.4.2 Finding volumes
With the Find button, you can search for a specific volume. The volume is specified with its
eight digit ID. See Figure 5-9.
Figure 5-9 Find Volume panel
If the desired volume is found, its designator is displayed in the upper-left hand corner. All
other volumes in the same LSS are also displayed.
5.4.3 Filtering volumes
In addition, you can filter the output of the volume display to a selected range by clicking the
Filter button and selection the Filter volumes option.
In the example shown in Figure 5-10 on page 135, we want to display Open System volumes
only. The volumes should be in simplex state; that means they are currently not in a copy
relationship.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
135
Figure 5-10 Filter volumes panel
5.4.4 Multiple Selection Mode
When you enter the Multiple Selection Mode, you can select multiple volume pairs when
you create tasks for both FlashCopy and PPRC. This mode is very useful to save time when
creating the tasks.
You might use this mode when selecting the volume for FlashCopy and PPRC tasks (for
further information, refer to 5.10.1, “Establishing a FlashCopy pair” on page 150, 5.11.3,
“Establishing a Synchronous PPRC pair” on page 167, and 5.11.4, “Establishing a PPRC
Extended Distance (PPRC-XD) copy pair” on page 173). Rather than defining one copy pair
at a time, you can define all the copy pairs at one time by selecting multiple sources and their
targets in the same task definition.
To enter the Multiple Selection Mode, click the Multiple Selection Mode button at the
bottom of the panel. This allows multiple pairs to be defined. Select the desired source
volume, and right-click the desired target volume. Repeat the selection of source and target
volumes until you have defined all volume pairs. Right-click one of the target volumes again
and the Task Wizard opens. You can decide whether to run one of the following operations:

Establish Synchronous PPRC copy pairs

Establish Extended Distance PPRC copy pairs

Suspend PPRC copy pairs

Terminate FlashCopy copy pairs

Establish FlashCopy copy pairs

Withdraw FlashCopy pairs
To exit the Multiple Selection Mode, click the Exit Multiple Selection Mode button at the
bottom of the Volume Panel.
Restriction: You can only perform the multiple selections on volumes within the same LSS
(all sources must be in the same LSS and all targets must be in the same LSS).

136
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.5 Logical Subsystems panel
The Logical Subsystems panel displays the Enterprise Storage Servers and the logical
subsystems (LSSs) within the storage network. The storage network includes all Enterprise
Storage Servers that are configured to use the same Copy Services server. Each of the
logical subsystems is specified by the serial number of the ESS it belongs to and its 2-digit
LSS number within the ESS.
With the Logical Subsystems panel, you will be able to:

Establish PPRC copy pairs for LSSs.

Convert PPRC Extended Distance copy pairs to synchronous pairs for all the volumes in a
source LSS and target LSS.

Filter a list of LSSs.

Find an LSS.

Freeze a PPRC Consistency Group.

Modify the PPRC Consistency Group time-out value.

Remove orphaned paths.

Resynchronize PPRC copy pairs for LSSs.

Run a consistency-group-created operation.

Suspend PPRC copy pairs for LSSs.

Terminate PPRC copy pairs for LSSs.

View information about an LSS.
In Figure 5-11 on page 137, you can see the Logical Subsystems panel for a selected ESS.
The color indicates the state of the LSS, whether it contains volumes that are currently in a
copy relationship (source, target, or mixed), or that they are not part of a copy pair at all.
Attention: If you click Exit Multiple Selection Mode before you perform any operations
on them, you lose all your previous selections.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
137
Figure 5-11 Logical Subsystem panel
Figure 5-2 explains the meanings of the icons that appear on the Logical Subsystems panel.
The Volumes panel displays similar icons for individual volumes (see Table 5-1 on page 132).
Table 5-2 Meaning of icons on Logical Subsystems panel
LSS Icon Appearance Meaning
Gray solid (“Simplex” state) All the volumes in the LSS are in simplex
state, meaning that they are not in a copy
relationship.
Blue solid (PPRC sync. source) All the volumes in the LSS are in PPRC pairs
(synchronous or Extended Distance) in a
non-suspended state.
Red solid (PPRC sync. target)

138
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.5.1 LSS Information Panel button
You can get more detailed information about a single logical subsystem by selecting the LSS
and clicking on the Information Panel button. In our example in Figure 5-12 we have
selected LSS 16, which is specified for open systems and contains 25 volumes. None of
these volumes are currently part of a FlashCopy or PPRC pair.
Figure 5-12 LSS Information panel
5.5.2 Finding LSSs
You can search for a specific LSS based on its address by selecting the Find button of the
Logical Subsystems panel. The LSS must be referred to using the following syntax: ESS
serial number:LSS number.
Figure 5-13 on page 139 shows an example in which we want to find the logical subsystem
12 of the Enterprise Storage Server with the Serial Number 22559.
Blue and white stripes (PPRC
suspended source)
All the volumes in the LSS are in PPRC pairs
(synchronous or Extended Distance) in a
suspended state.
Red and white stripes (PPRC
suspended target)
Yellow solid Not all volumes in the LSS are in the same
type of Copy Services relationship (this apply
for PPRC and FlashCopy relationships), and
none of the volumes are in a suspended
state.
Yellow and white stripes (mixed
types and states)
The volumes in the LSS are either the same
type but some are in a non-suspended state
or not all of the volumes are the same type but
some are in suspended state.
LSS Icon Appearance Meaning

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
139
Figure 5-13 Find LSS
5.5.3 Filtering LSSs
In addition, you can limit the output of the volume display to a selected range by clicking the
Filter button and selecting the Filter devices option.
In our example (Figure 5-14), we want to display physical and logical storage servers only. In
addition, we only want to display open systems devices that contain volumes that are
currently part of a Copy relationship.
Figure 5-14 Filtering logical subsystems
5.5.4 Properties
By selecting one LSS and clicking the Properties button, you can view or change the copy
properties of the entire LSS (see Figure 5-15 on page 140).

140
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-15 LSS properties panel
PPRC Consistency Group
To maintain consistency of data across volumes at the recovery site, volumes in a PPRC
relationship can be collected into a
Consistency Group
. If you mark this check box, than all
the volumes in that LSS will be considered as part of one Consistency Group. Creating a
Consistency Group of volumes provides the ability to temporarily queue subsequent write
operations to all PPRC Consistency Group volumes on an single LSS pairing when an error
occurs on one of the volumes in the group (primary or secondary), or when a total link failure
is detected between the primary and secondary LSS pair. The other way to establish a
Consistency Group is to select the PPRC Consistency Group option when establishing a path
between two LSSs. For further information about PPRC Consistency groups, refer to 4.5,
“Asynchronous PPRC” on page 74.
XRC Session Time Out
This parameter does not apply to open systems.
CONCOPY Session Time Out
This parameter does not apply to open systems.
PPRC Consistency Group Time Out
This parameter indicates the amount of time that an I/O is withheld from updating a primary
volume of a Consistency Group in an case of an error event. This timeout enables automation
software to detect that an error has occurred and to issue commands to
freeze
all other
members of the Consistency Group. When the Consistency Group is created, this parameter
is set to a default of two minutes. For further information about PPRC Consistency Groups,
refer to 4.5, “Asynchronous PPRC” on page 74.
5.6 The Paths panel
A path is used to send data between the primary and secondary volumes of PPRC pairs.
The physical path consists of the ESCON connection between two Enterprise Storage
Servers, while a logical path describes the connection of the PPRC source and targets.
Be aware that before you can create PPRC copy pair relationships, you must use the Paths
panel to establish logical paths between logical subsystems between the source and the
target LSS. After you establish the paths, the Paths panel displays the current status of the

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
141
paths for the selected LSS. There could be multiple logical paths established over a single
physical path. This is recommended for availability and performance reasons. For further
information, refer to 4.9, “PPRC connectivity” on page 101.
From the Paths panel, you will be able to:

Establish paths.

Add paths.

Remove a group of established paths.

Remove one or more paths from a group of established paths.

Create a PPRC Consistency Group.

View information about paths.
Figure 5-16 shows the entry panel of the Paths panel.
Figure 5-16 The Paths panel
In the upper right corner of the path panel from the drop-down menu, Select Source
Subsystem, you select the primary LSS, which is denoted by an ESS serial number and LSS
number. In this example, we use LSS 11 of the ESS with the serial number 22559. This
shows you all configured ESCON adapters of the primary ESS in the Path Origin column of
the Paths panel and their targets. Table 5-3 on page 142 gives you an overview about the
connection symbols used in the “Local port / remote port” column located on the left side of
the panel.

142
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Table 5-3 Meaning of the Paths panel icons
The ESCON adapters are specified by their System Adapter ID (SAID). Figure 5-17 shows
the SAID of all ESS ESCON adapters (see 4.9, “PPRC connectivity” on page 101 for further
information).
Figure 5-17 SAID numbers of the ESS ESCON adapters
Once an ESCON adapter is selected, all Enterprise Storage servers that are connected to
this adapter are displayed in the Common Storage Server Targets column. All logical
subsystems that are available on a particular ESS will be listed in the Logical Subsystem
Targets column if one of the Storage Servers is selected.
Connection icon Description
Host adapter port, with the system adapter identification number
(SAID) provided under it.
ESCON director.
Host server.
Storage server.
SAID 0000
SAID 0001
SAID 0004
SAID 0005
SAID 0008
SAID 0009
SAID 000C
SAID 000D
SAID 0020
SAID 0021
SAID 0024
SAID 0025
SAID 0028
SAID 0029
SAID 002C
SAID 002D
SAID 0080
SAID 0081
SAID 0084
SAID 0085
SAID 0088
SAID 0089
SAID 008C
SAID 008D
SAID 00A0
SAID 00A1
SAID 00A4
SAID 00A5
SAID 00A8
SAID 00A9
SAID 00AC
SAID 00AD
BAY1 BAY2 BAY3 BAY4
SAID numbers of the ESS ESCON adapters
Host adapter bays viewed from the frontside of the Enterprise Storage Server

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
143
In the example shown in Figure 5-18, we have selected the ESS 2105:14146:17, that is, the
LSS number 17 of the ESS with serial number 14146. The path origin is the ESCON adapter
with the SAID 0200. The selected target is ESS 2105:13350.
Figure 5-18 Example of the Paths panel
5.6.1 Display Direct Connection Paths
If you have two ESSs that are directly connected (that is, no switch between them), you can
click Display Direct Connect Paths to display those paths.
5.6.2 Path Information Panel button
Once an ESCON adapter is selected, you can get more information about the paths by
clicking the Information Panel button at the bottom of the Paths panel.
The example shown in Figure 5-19 on page 144 shows a path defined between source LSS
10 and target LSS 11 using the ESCON adapters SAID0000 and remote SAID0080 on the
same ESS.
Note: You must perform a refresh function for the following conditions:

If this is the very first time anyone is accessing the ESS Copy Services WUI.

After an update of the ESS microcode.

After a concurrent service action.

If the cabling for the ESCON links is modified.

144
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-19 Path Information panel
If there are logical paths defined on an ESCON adapter, you will find three blue asterisks right
below the adapter in the Path Origin column. Three red asterisks below the path-connection
symbol in the Path Origin column means that the last attempt to establish the path failed.
Figure 5-20 shows an ESCON adapter SAID without, then with, the defined path, and finally
with a failure to establish the path.
Figure 5-20 ESCON adapter without, with defined path, and failed establish path
5.7 Tasks panel
With the ESS Copy Services, you have the possibility to save the setup of any data copy
action within a Task. This could be any kind of FlashCopy, PPRC, and path operation.
With the Tasks menu, you will be able to:

Group tasks.

Modify a task.

Remove a task group.

Remove a saved task.

Run a saved task.

Ungroup tasks.

View error information about a failed task.

View information about a saved task.
Figure 5-21 on page 145 shows the Tasks panel of the ESS Copy Services. For each task,
the name, a description, and the last status of the execution is displayed.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
145
You will be prompted to choose a task name when you create a task. Use the following
guidelines:

The name can only contain alphanumeric characters.

The name cannot contain blanks.

The name can contain the underscore (_) or hyphen (-) characters.

The name cannot exceed 16 characters.
Figure 5-21 The Tasks panel
5.7.1 Grouping and ungrouping tasks
To create a group task, click the single tasks you want to group together while holding the
Shift key or the Control key (mark single tasks). Once you are finished, click the Group button
and specify the group name. It is not possible to include a group into another task group, all
tasks within a task group will be processed in parallel.
An example for the usage of a task group would be multiple FlashCopy pairs from different
logical subsystems that need to be issued all at the same time in order to do a backup.
In our example in Figure 5-22 on page 146, we have a grouped task named Group_task. This
group contains two single tasks named CLI_task5 and CLI_task6. Both of the tasks establish
a FlashCopy pair within LSS 00.
Note: In Appendix E, “Sample naming convention” on page 561, we present the task
naming convention used in this chapter. You can refer to this appendix to find some hints
and tips about naming conventions.

146
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
You can get detailed information about the setup of the task. Select the task and click the
Information Panel button at the lower right.
Grouping tasks will cause the individual tasks to collapse into the overall group task.
Figure 5-22 Task information panel
To ungroup a task, select the task and click the Ungroup button at the bottom of the Tasks
panel. It is necessary to ungroup the overall group task in order to gain access and to execute
the individual tasks.
5.7.2 Removing a task
To remove a task, select the task and click the Remove button at the bottom of the Tasks
panel.
5.7.3 Running a task and viewing an error about a failed task
To run a saved task, select the task and click the Run button at the bottom of the Tasks panel.
The task will be executed immediately.
If the task fails, a failure report is available through the Information panel. The failure report
contains the error messages and the sense data. The messages are documented in the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448, which is
available on the ESS Web site.
You can also view the logs in the administration panel.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
147
5.7.4 Modifying a task
Use the Tasks panel to modify a previously saved task. You can change the task goals, task
options, task name and description.
You will be prompted if you want to run, to replace the selected task, or create a new task.
Use the Tasks panel to modify a previously saved task. You can change a task from one that
establishes a PPRC copy pair to one that suspends, re-synchronizes, or terminates a PPRC
copy pair. For FlashCopy, you can change a task that establishes a FlashCopy pair to one that
withdraws a FlashCopy pair. Or you can change the options of one task.
5.8 Administration panel
Use the Administration panel (see Figure 5-23) to manage the server logs, reports, and ESS
Copy Services Command Line Interface (CLI) user IDs (user IDs) and passwords. You can
also use this panel to refresh the volume and LSS information for an ESS.
Figure 5-23 Administration panel
You can perform the following actions from the Administration panel:

Clear the server logs.

Manage the CLI user ID and password for an open-systems host:
– Define the user ID and password.
Note: Grouped tasks cannot be modified. The tasks must be ungrouped and then the
tasks can be modified.

148
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
– Remove the user ID and password.

Refresh the path and volume information for an ESS.

Send the ESS Network Configuration report to an e-mail address.

Send the ESS Resource Configuration report to an e-mail address.

Send the server logs to an e-mail address.

Specify the e-mail address that receives server logs, ESS Network Configuration reports,
and ESS Resource Configuration reports.

View the ESS Network Configuration report.

View the ESS Resource Configuration report.

View the server logs.
5.8.1 Working with the Copy Services logs and reports

The Copy Services server maintains the following logs and reports:
– Copy Services status log (copyservices.log): This contains messages that are
associated with user actions issued through ESS Copy Services. This log includes
messages associated with such actions as saving tasks and executing tasks. The log
also includes information about the success or failure of the tasks. You can click the
error message to get a description of the problem and its possible solutions.
– Copy Services Timing log (rsCStiming.log): This contains information about the
time it took for data-copy functions (PPRC or FlashCopy) to complete on a specific
volume.
– ESS Network Configuration report: The ESS Network Configuration report contains
important information about your network-defined ESS resources:
• IP addresses for your primary and Backup Copy Services servers
• IP addresses and host names for the ESS clusters
• IP addresses for the ESS clients
– ESS Resource Configuration report: The ESS Resource Configuration report
contains the following important information about your ESS resources:
• The 2105 connection information (SAIDs, port types, and values)
• PPRC path information (SAIDs, LSSs, remote SAIDs, ports, and status)
• S/390 or zSeries volumes (LSSs, labels, Concurrent Copy status, FlashCopy
status, PPRC status, or XRC status)
• Open-systems volumes information (LSSs, volume serial numbers, user defined
label, FlashCopy status, and PPRC Status)
To clear a log, select the log in the Log/Report list and click the Clear button. Notice that you
cannot clear a report. You can view the log or report by selecting the appropriate log/report
and clicking the View button.
Attention: The status log is useful for problem determination. Therefore, you should
only clear it when no problems exist or when you have completed data collection for
all failures.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
149
To send a report or a log to an e-mail address, select the appropriate log/report in the
Log/Report list and click the E-Mail button. Notice that you have to specify the e-mail
address in the E-Mail field.
5.8.2 Refreshing path and volume information for an ESS
Use the refresh function of ESS Copy Services to get the most up-to-date information about
paths and volumes for an ESS. To perform this action, select the ESS for which you want this
information in the ESS list and click the Refresh button.
A status message opens on the top of the Administration panel noting that:
A refresh request was sent to <the selected ESS>. It may take a few minutes before the
server can refresh the data.
5.8.3 Managing the CLI user ID and password for an open-systems host
Use the administration panel to manage the user IDs (user IDs) and passwords for the Copy
Services Command Line Interface (CLI). Open-systems hosts that issue CLI commands to a
Copy Services server must provide a valid user ID and password as part of the command
string. If the Password Protection option is enabled and the hosts do not provide the user ID
and password, the command fails.
To enable the password protection for access to an ESS through the CLI, click Enabled in the
Password protection field. To add users, click the Users button and the Authorized Users
dialog box will appear. Click the Add button and enter the user name and password. If you
have entered the user name and the password, click the Add button and the new user will be
authorized. To remove a user, select the user in the Authorized Users dialog box and click the
Remove button. After you have performed all actions, click Done and the Authorized Users
dialog box will be closed.
5.9 Exiting ESS Copy Services
There are several legitimate ways to quit the ESS Copy Services Web User Interface:

You can click the X at the top right corner of the ESS Copy Services panel.

You can click Exit in the navigation frame of any of the ESS Copy Services panels.

You can shut down the whole browser from another browser window.
In addition, if you simply want to return to the ESS Launch page without closing ESS Copy
Services, such as to launch ESS Specialist, you can keep the ESS Copy Services browser
window open and use standard navigation methods such as Alt-Tab to switch to the ESS
Launch page.
The difference between the first two options above is nothing more than a warning message.
If you click Exit in the navigation frame of any of the ESS Copy Services panels, the warning
message opens on top of the browser window and tells you that the tasks that you have
Attention: When you define the user ID and password, the active Copy Services server
maintains them. The active Copy Services server does not send them to the Backup Copy
Services server as it does for Copy Services tasks. Therefore, you must define the user ID
and password at the Backup Copy Services server in a separate step. You must manually
restart the Backup Copy Services server before you can define the user ID and password
there. See 9.4, “Copy Services server failover” on page 461.

150
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
submitted will continue running after you close the window. Click OK. The ESS Copy Services
browser window closes and the ESS Launch page comes into focus. In either case, the next
time you click Copy Services on the ESS Launch panel, the ESS Copy Services browser
window opens without requiring you to log in. The browser loads a cached version of the
applet.
5.10 Performing FlashCopy with the Web User Interface
In this section, we explain how to set up FlashCopy relationships using the ESS Copy
Services Web interface.
Be aware of these requirements of the FlashCopy functionality:

The source and target volume have to be in the same LSS.

The target volume must be the same size as the source volume or larger.

A volume can be in only one FlashCopy relationship at a time.
There are two different ways of establishing a FlashCopy pair:

From the Volume panel

From the Tasks panel (once a task for a FlashCopy is created and saved)
5.10.1 Establishing a FlashCopy pair
Use the Volumes panel to establish a FlashCopy pair. Select the LSS within which you want
to perform the FlashCopy. This can be either done in the source or target area of the volumes
panel (Figure 5-24 on page 151).
Note: Loading the cached applet is a problem in two situations:

If you load a new version of the ESS microcode during that time while you have not yet
quit your browser, the cached applet might not conform with the new code on the ESS.

If unauthorized users have access to your workstation when you leave it, they can
access the ESS through your cached user ID.
Note: Prior to establishing a FlashCopy, we recommend that you do the following:
1.Unmount the target volume from all host systems, since the FlashCopy process is a
destructive to the target and will overwrite the data on the target volume.
2.Additionally, stop all active reading/writing operations to the target.
3.Also, quiesce your application and flush the data to the source volume.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
151
Figure 5-24 Volumes panel
You always need to have two components to establish a FlashCopy pair: a source and a
target. Select the source volume, and, with a right-click, the target volume. If you have
selected the wrong source or target volume, just click on the correct source volume again.
Once you have selected the source and the target, you do a second right-click on the target to
bring up the Task Wizard (Figure 5-25). Select the Establish FlashCopy pair option and click
Next.
Figure 5-25 Task Wizard panel
Within the next window, you can specify the Copy option of the FlashCopy pair (Figure 5-26
on page 152). For further information about this option, refer to 3.4, “FlashCopy tasks and
options” on page 41.

152
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-26 Select copy options panel
Selecting the FlashCopy Options
3.4, “FlashCopy tasks and options” on page 41 gives detailed descriptions of the different
options for FlashCopy. Refer to this section to be sure to fully understand the different options.
We present a summary here of the different options.
No background copy option (nocopy)
If this option is checked:

Only the tracks that are modified on the source volume are copied to the target volume.

This relationship between source and target volume remains forever and has to be broken
manually.

By default, this option is not selected and all data is copied from the source to the target
volume of a FlashCopy pair (background copy). Once all data is copied, the relationship
ends automatically (unless the Persistent FlashCopy option is selected).

If two volumes are in a FlashCopy
nocopy
relationship, you can perform a FlashCopy Start
Background Copy to create a physical background copy.
Accelerate destage mode
Select this option to cause a FlashCopy source volume track, which is being modified and is
in FlashCopy relationship, to be destaged from cache sooner than it would be if normal cache
algorithms were applied. This feature minimizes (for the source volume) the number of
modified tracks that are resident in cache.
Note that:

With FlashCopy Version 1, this option might (under specific circumstances) speed up the
completion of the background copy process.

With FlashCopy Version 2, This feature has no effect, since the cache algorithms have
changed.
Permit establish if target is online
Select this option if you want to establish a FlashCopy relationship even if the target is online
to the S/390 or zSeries host.
This option does not apply to open-systems volumes.
Persistent FlashCopy
If you want the FlashCopy relationship to remain after the FlashCopy operation completes
you can use the persistent FlashCopy option. The FlashCopy relationship between source

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
153
and target volumes remains indefinitely and must be broken by a Withdraw FlashCopy task.
By default, this option is not selected.
From the next panel you can either Save, Run, or Cancel the copy task, as shown in
Figure 5-27. Once a task is saved, it can be executed from the Task panel at any time.
Optionally, a name and description of the task could be specified. Even if you do not want to
save the task you have created, we recommend specifying a name and description. This will
help with the interpretation of the Copy Services log file later on. An example would be to
retrieve the execution time of the background copy of a FlashCopy pair.
Figure 5-27 Define Task panel
If a FlashCopy is issued, a bitmap is created for the data copy from the source to the target.
The time to establish the FlashCopy relationship is only a few seconds. After this period, the
source is immediately available to the host system, and data will be copied from the source to
the target in the background.
Once a FlashCopy is started, the display of the source and target volume from the Volumes
panel changes:

A blue lightning bolt appears on the FlashCopy source volume.

A red lightning bolt appears on the FlashCopy target volume.

In addition, a FlashCopy copy icon (a small white square on the lower left of the volume
icon (see Table 5-1 on page 132) now also appears on the target volume at the time that
the copy begins. The FlashCopy copy icon remains until you issue a command to withdraw
it (see 5.10.3, “Withdrawing a FlashCopy pair” on page 155), or until you use the volume
as a source volume.

The lightning bolts disappear when the copy operation completes. If the pair was
established without a background copy, the lightning bolts remain until the withdraw
operation is performed or until the entire volume is rewritten.

The Information panel shows the progress of the copy.

154
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-28 FlashCopy volume display
Establishing FlashCopy relationships using the LSS panel
To establish FlashCopy relationships between all the volumes in two LSSs, you can create a
task at the LSS level in the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystems panel
do have different meanings (for further information, refer to Table 5-2 on page 137).
Click on the source LSS and the LSS ID turns blue. Right-click the target LSS and the LSS ID
turns red. Right-click again on the target LSS and the Task Wizard opens. Select the options
as you would do for volumes pairs.
5.10.2 Getting Information about a FlashCopy pair
By selecting one of the volumes of a FlashCopy pair and clicking the Information button, you
get information about this particular pair. If you have selected the source volume, you can see
how many tracks still have to be copied to the target volume (Figure 5-29 on page 155).
Tip: After you have created and saved the task to establish the FlashCopy with the
No background copy option, you need to create the corresponding task to withdraw the
pair. We recommend that you modify the task that you just created so that it withdraws the
pair, then name and save the task. This not only saves time, but it is also useful for Disaster
Recovery. For further information, refer to 5.7.4, “Modifying a task” on page 147.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
155
Figure 5-29 Information panel of a FlashCopy source
5.10.3 Withdrawing a FlashCopy pair
In the following cases, you have to
withdraw
a FlashCopy pair:

If a FlashCopy pair is not needed anymore, but has not yet finished the background copy.

If a FlashCopy pair that was created with the Do not perform background copy option is
not needed anymore.

If a FlashCopy pair that was created with the Persistent FlashCopy option is not needed
anymore.
The command to Withdraw FlashCopy pair results in the termination of the FlashCopy
operation. All the data on the target can be corrupted and unusable if it is inadvertently written
on by another application.
Use the Volumes or the Logical Subsystem panel to withdraw a FlashCopy pair.
Withdrawing FlashCopy pairs using the Volumes panel
Open the Volumes panel and select the LSS within which you want to perform the withdraw
FlashCopy. This can be either done in the source or in target area of the volumes panel
(Figure 5-30 on page 156).

156
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-30 Volumes panel
You always need to have two components to withdraw a FlashCopy pair, a source and a
target. Select the source volume and, with a right-click, the target. If you have selected the
wrong source or target volume, just click on the correct source volume again.
To withdraw a FlashCopy pair, select the source volume, select the target volume with a
right-click, and start the Task Wizard (Figure 5-31) by right-clicking the target volume again.
Figure 5-31 Withdraw a FlashCopy pair
Select the Withdraw FlashCopy pair option and decide whether to Save, Run, or Cancel
the task (Figure 5-32 on page 157).

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
157
Figure 5-32 Define Task panel
If you save the task, it can be executed from the Tasks panel any time. Once you have run the
task, the lightening bolts on the source and target volume disappear, and the FlashCopy
relationship between source and target volume will be removed. Also, the FlashCopy icon
(see Table 5-1 on page 132) on the target volume will disappear.
Withdrawing FlashCopy pairs using the LSS panel
To withdraw all FlashCopy pairs between two logical subsystems, you can create a task at the
LSS level, in the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystems panel
do have different meanings (for further information, refer to Table 5-2 on page 137).
Click on the source LSS and the LSS ID turns blue. Right-click on the target LSS and the LSS
ID turns red. Right-click again on the target LSS and the Task Wizard opens. Select the
options as you would do for volume pairs.
Withdraw FlashCopy to the target
Use the option Withdraw FlashCopy to the target against the target volume when you want
to remove its FlashCopy icon (represented by a white square at the lower left of the volume
icon; see Table 5-1 on page 132) after the background copy has been successfully
completed. Typically, you would remove the indicator because you have decided that it is no
longer necessary to mark the volume as a copy of another.
Note: All the FlashCopy relationships that have source volumes in the selected source
LSS and their associated targets in the selected target LSS will be terminated.
Attention: If you perform a Withdraw to target action to a target volume of an established
relationship, it will withdraw the relationship and remove the “I-am_a_target” icon.

158
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To perform this action, click on the target volume, right-click on the target volume (the volume
ID turns gray), and right-click the target volume again to open the Task Wizard (Figure 5-33).
Figure 5-33 Task Wizard: Withdraw FlashCopy pair
Select the Withdraw FlashCopy pair option if you want to immediately terminate the
FlashCopy relationship and click Next. In the next panel (Figure 5-34) select the FlashCopy
withdraw to the target option. Click Next and decide whether to Save, Run, or Cancel the
task (Figure 5-32 on page 157). Once you have run the task, the FlashCopy icon (see
Table 5-1 on page 132) on the target volume will disappear.
Figure 5-34 Task Wizard: Withdraw options
Withdrawing FlashCopy to the target using the LSS panel
To withdraw all FlashCopy relationships to the target for all target volumes in the same logical
subsystem, you can create a task at the LSS level, in the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystems panel
do have different meanings (for further information, refer to Table 5-2 on page 137).
Note: If you perform the Withdraw FlashCopy to the target, make sure you select a
target volume of a FlashCopy relationship in the volumes panel. You cannot perform this
action by only selecting a source volume.
Note: All FlashCopy relationships that have a target volume in the LSS you select will be
terminated.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
159
Left-click on the LSS and the LSS ID turns blue. Right-click the same LSS and the LSS ID
turns gray. Right-click again on the LSS and the Task Wizard opens. Select the options as you
would do for volumes pairs.
5.10.4 FlashCopy Start Background Copy
If you have performed a FlashCopy pair with the No background copy option and you need
to create a permanent physical copy of the data for backup or disaster recovery purposes,
you can perform a FlashCopy Start Background Copy. All data will be copied from the
source to the target. After the copy is complete, the FlashCopy relationship is terminated,
unless the FlashCopy relationship is persistent, in which case, the relationship remains.
FlashCopy Start Background Copy using the Volumes panel
Open the Volumes panel and select the LSS within which you want to perform the
FlashCopy Start Background Copy. This can be done in either the source or target area of
the volumes panel (Figure 5-35).
Figure 5-35 Volumes panel
To perform a FlashCopy Start Background Copy, click on the source volume of the
FlashCopy pair. Right-click the source volume (the target ID of the volume turns gray) and
right-click the volume again to open the Task Wizard (Figure 5-36 on page 160).

160
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-36 Task Wizard: Withdraw FlashCopy pair
Select the Withdraw FlashCopy pair option and click Next. In the next panel, select the
FlashCopy Start Background Copy option (Figure 5-37).
Figure 5-37 Task Wizard: Withdraw options
Click the Next button and decide whether to Save, Run, or Cancel the task. Once you run the
task, the background copy task copies the data from the source to the target. After the copy is
complete, the FlashCopy relationship is terminated and the lightning bolts disappear on the
source and the target volumes, unless the FlashCopy relationship is persistent, in which case,
the relationship remains. Refer to 5.10.1, “Establishing a FlashCopy pair” on page 150 for
further information.
FlashCopy Start Background Copy using the LSS panel
To start a background copy of all FlashCopy pairs that have a source volume on a logical
subsystem, you can use the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystem Panel
do have different meanings (for further information, refer to Table 5-2 on page 137).
Note: If you perform FlashCopy Start Background Copy, make sure you select a source
volume of a FlashCopy relationship in the volumes panel. You cannot perform this action
by only selecting a target volume.
Note: When you run the task at the LSS level, be aware that the task will run on all the
source volumes in that LSS.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
161
Click on the desired LSS. The LSS ID turns blue. Right-click the LSS and the LSS ID turns
gray. Right-click the LSS again and the Task Wizard opens. Select the options as you would
do for volumes pairs.
5.11 Performing PPRC with ESS Copy Services Web User Interface
In this section, we explain how to set up PPRC and PPRC-XD using the ESS Copy Services
Web User Interface. For further information about PPRC and PPRC-XD, refer to Chapter 4,
“Peer-to-Peer Remote Copy (PPRC)” on page 55. In general, there are two steps needed to
successfully establish PPRC:

Setting up paths between the PPRC source and target.

Establishing, suspending, and terminating the PPRC pairs, either single volumes or entire
logical subsystems.
Be aware of these requirements of the PPRC functionality:

Paths for PPRC have to be available and need to be defined first.

All PPRC ESCON links are unidirectional.

The target volume has to be the same size as the source or larger.

You can have up to four ESSs in a Copy Services group.
5.11.1 Establishing paths
Before you can establish any PPRC pairs, you first have to set up the paths between the
source and the target LSSs. The paths are needed for communication between the PPRC
pairs and to copy data from the source to the target.
Use the Paths panel of the ESS Copy Services Web Interface to set up paths for PPRC
(Figure 5-38 on page 162).
Note: To set up all PPRC pair related functions, either the Volumes panel or the Logical
Subsystems panel (all volumes in the source and the target LSS) can be used. In this
book, only the usage of the Volume panel is explained. For further information about how
to set up a PPRC pair related function on the Logical Subsystem panel, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.
Note: In our example we have used only one ESS to set up PPRC. That is possible, as the
ESS contains both source and target volumes at the same time. However, for high
availability and disaster recovery configurations, two or even more Enterprise Storage
Servers are required.

162
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-38 Paths panel of the Copy Services Web User Interface
Select the source of the PPRC relationship. This is done with the drop-down menu of the
Select box. All available ESCON adapters for the source will automatically be displayed in the
Path Origin area (see Figure 5-38).
Select the ESCON adapters you want to use for the PPRC.
The ESSs that are connected to the adapter(s) will be automatically displayed in the Common
Storage Server Target area.
Note: Multiple adapters could be selected with a right-click after the first ECSON adapter
was selected. If you choose the wrong adapter(s), just click on the correct ESCON adapter
again to delete the selection.
Note: If you have some ESSs that are directly connected (that is, no ESCON switch
between them), you can click Display Direct Connect Paths to display those paths.
You must perform a refresh function (see 5.6.1, “Display Direct Connection Paths” on
page 143) for the following conditions:

If this is the first time you are accessing ESS Copy Services

After an update of the ESS microcode

After a concurrent service action

If the cabling for the ESCON links is modified

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
163
Next, click on the target ESS in the Common Storage Server Target area. All logical
subsystems available on the target ESS will be displayed in the Logical Subsystems
Targets area (see Figure 5-39).
Within the Logical Subsystems Targets area, select the target LSS of your PPRC path. If you
have chosen the wrong target, just click on the correct LSS again to delete the selection.
In the example shown in Figure 5-39, we have selected LSS 11 to be the target logical
subsystem.
Figure 5-39 Establish PPRC path
Once the target and source of the PPRC path have been selected, click on one of the
highlighted target LSSs to bring up the Task Wizard (Figure 5-40). Select the Establish Path
option and click Next.
Figure 5-40 Task Wizard: Establish Path

164
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
If a switch is part of the path, the Select outgoing ports panel of the Task Wizard opens
(Figure 5-41).
Figure 5-41 Task Wizard: Select outgoing ports panel
In the Outgoing Ports list, select the desired port on the switch. Click Next. Within the next
panel, you can specify the path options (Figure 5-42).
Figure 5-42 Establish PPRC options

Do not establish paths if they already exist: If this option is checked and there is
already a path defined from the source to the target, the operation of establishing paths
will not be executed.

Force removal of existing paths: You must select this option if the selected path goes
through a switch. The selection causes any logical paths between a host system and the
selected ESCON adapter to be removed before establishing logical paths.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
165
In either case, you can also select PPRC Consistency Group to create a Consistency
Group. For information on Consistency Groups, see 4.5, “Asynchronous PPRC” on page 74.
Click Next and decide whether to Save, Run, or Cancel the task. Click Save if you want to
save the task in the task repository. If you save the task, it can be executed form the Tasks
panel any time.
When you run the task, the PPRC path is established and three blue asterisks appear right
below the adapter in the Path Origin column of the Paths panel (see Figure 5-43).
Figure 5-43 Path successfully established
5.11.2 Removing paths
If you do not need a path any more, you can remove it. Use the Paths panel to remove a path.
From the Select Source Subsystem list, select the source logical subsystem. The blue
asterisks below the path-connection symbol in the Path Origin column mean that a path is
already established. Select the desired path in this column. The Paths panel displays the
common storage servers and their designators (Figure 5-44 on page 166).
Attention: The Force removal of existing paths option removes all logical paths from
the host to the port. This is potentially disruptive to host activity, since there may be
applications using the port. It is recommended that you vary off all logical paths from
the host to the port by means of a host command. Only as a last resort, or when the
operator is absolutely positive there would be no negative impact, should the Force
removal of existing paths option be used.
Tips:

If you save the task, it can be executed from the Tasks panel at any time (see 5.7.3,
“Running a task and viewing an error about a failed task” on page 146).

After you create and save the task to establish paths, you need to create the
corresponding task to remove the path. You can modify the task within the Tasks panel
so that it removes the path (see 5.7.4, “Modifying a task” on page 147). This not only
saves time, but it is also useful for Disaster Recovery.

166
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-44 Paths panel: Remove paths
Select the desired ESS in the Common Storage Servers Targets column and the Paths
panel displays a list of logical subsystems. Select the logical subsystem that is the destination
path and right-click on the desired logical subsystem (Figure 5-44). The Task Wizard opens
(Figure 5-45).
Figure 5-45 Task Wizard: Remove paths
Select the Remove paths option and click Next. In the next panel, you can specify the task
options (Figure 5-46 on page 167).

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
167
Figure 5-46 Task Wizard: Remove tasks options

Do not remove PPPC path if pairs exists: If you choose this option, the path will not be
removed if there is a PPRC pair using this path.

Force removal of PPRC path even if pairs exists: This option enables you to force the
removal of existing PPRC paths even if there are PPRC pairs established that could use
this path. This option is helpful for the failover/failback mode. For further information, refer
to 9.3, “Types of PPRC Failover and Failback procedures” on page 460.
From the next panel, you can either Save, Run, or Cancel the task. As mentioned before,
once a task is saved, you can run it from the Tasks panel at any time.
Once you have run the task, all PPRC paths from the source to the target LSSs will be
removed and the blue asterisks below the Path Connection symbol in the Path Origin column
will disappear.
5.11.3 Establishing a Synchronous PPRC pair
Use the Volumes panel to establish PPRC pairs. On the left side, you select the source LSS;
and on the right side, the target LSS. This is done using the drop-down menus at the top of
the Volumes menu.
The source and target logical subsystem is specified the following way:
Device type (4 digits):ESS Serial number (5 digits):LSS number (2 digits)
In our example shown in Figure 5-47 on page 168 we have selected 2105:22547:10 as
source and 2105:22549:11 as target of our PPRC pair.
Tip: To remove a path, you can either use the Paths panel, the Tasks panel (by modifying
an Establish Paths task or by running a Remove Paths task) or by using the Logical
Subsystems panel. For further information, refer to the IBM TotalStorage Enterprise
Storage Server Web Interface User’s Guide, SC26-7448.

168
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 5-47 Volume panel: establish PPRC pair
You always need to have two components to establish a PPRC pair: a primary (source) and a
secondary (target). Select the source volume, and, with a right-click, the target. If you have
selected the wrong source or target volume, just click on the correct source volume again to
clear the selection (Figure 5-47).
Once you have selected the primary and the secondary, you do a second right-click on the
secondary (target) to bring up the Task Wizard (Figure 5-48). Select the Establish PPRC
copy pair option and click Next.
Figure 5-48 Task Wizard: Select task type
Within the next panel, you can specify the Copy options of the PPRC pair (Figure 5-49 on
page 169). Click Next when you have finished the selection.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
169
Figure 5-49 PPRC copy options
Options for PPRC synchronous

Copy initialization:
– Do not copy volume: If this option is checked, the PPRC pair relationship is
established without copying any data from the source to the target. This option is used
when the source and target contain exactly the same data and are consistent while in
the simplex state. Choosing this option causes the volumes to turn duplex immediately
on the assumption that both source and target contain identical data.
– Copy entire volume: If this option is checked, all data is copied from the source to the
target volume. This option has to be used the first time a PPRC relationship is going to
be established and is needed to guarantee that source and target contain the same
data.
– Copy out-of-sync cylinders only: This option copies only the data that was updated
on the source volume since a PPRC copy pair was suspended. The option is used to
resynchronize a PPRC synchronous or a PPRC-XD pair.

PPRC modes:
– PPRC Failover: Select this option to reverse the direction of a PPRC pair. Use this
option to make your current secondary volume at the recovery site become a primary
volume (your current primary volume at the production side being the secondary
volume of the PPRC relationship). For further information, refer to Chapter 9, “Disaster
Recovery” on page 453.
– PPRC Failback: Select this option to resynchronize the PPRC volumes at the
recovery site with their peer volumes at the production site. For further information,
refer to Chapter 9, “Disaster Recovery” on page 453.

Copy options:
– Permit read from secondary: Selecting this option allows host servers to read from
the PPRC secondary volume. The PPRC pair must be in a full-duplex state in order for
the host server to read the volume. This option is helpful for operating systems that use
a Logical Volume Manager that reads data structures from the header of a secondary
volume to update internal databases, such as the ODM under AIX, or /etc/lvmtab under

170
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
HP-UX. For further information, refer to Chapter 8, “Open systems specifics” on
page 407.
– Suspend PPRC after establish complete: If you select the Copy entire volume or
Copy out-of-sync cylinders only options described above, this means that after the
data is completely copied to the target volume, the target volume goes into the
suspended state. Selecting this option saves you the effort of creating a separate task
to suspend the pair. This is useful in a PPRC-XD catch-up operation over extended
distances.
From the next panel, you can either Save, Run, or Cancel the copy task, shown in
Figure 5-50. Click Save to run the task and run it later. To run the task immediately, click Run.
Figure 5-50 Task Wizard: Establish PPRC path
Once a Synchronous PPRC pair is established, the display of the primary and secondary
volumes from the Volumes panel changes, indicating that the initial copy is in progress
(Figure 5-51 on page 171).
Tip: After you create and save the task to establish Synchronous PPRC copy pairs, we
recommend that you create the tasks that suspend the pairs and terminate the pairs. You
can save time in creating tasks if you use the task you just saved as a template to create
the suspend task. Then use the suspend task as a template to create the terminate task. In
the Task panel, select the task you created and click on the Modify button to change the
options of the task. Then, give a new name and save as a new task.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
171
Figure 5-51 PPRC relationship in progress

172
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Once the volumes are in sync, the icons of the volumes icons will change to solid
(Figure 5-52).
Figure 5-52 PPRC pair in sync
Select a volume and click the Information Panel button to retrieve more information about
the status. If the source of a PPRC pair is selected, the number of out-of-sync cylinders that
are still left to copy are displayed. Those are the tracks that need to be copied from the source
to the target to achieve full copy mode (Figure 5-53 on page 173).

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
173
Figure 5-53 Information panel of PPRC source volume
5.11.4 Establishing a PPRC Extended Distance (PPRC-XD) copy pair
PPRC Extended Distance provides the following advantages over the standard Synchronous
PPRC mode:

Minimal response-time delays for application hosts that are writing to a PPRC primary
device

Copy distances that can be increased by thousands of miles with channel extenders

Data migration without the response-time penalty of the Synchronous PPRC copy mode

No FlashCopy of the primary volume required to eliminate the synchronous-write penalty
To establish the PPRC Extended Distance copy pair, use the Volumes panel. This procedure
is almost identical to the previous one, which you can use to create a Synchronous PPRC
copy pair, except that you select the Establish Extended Distance PPRC option in the
Select task type panel of the Task Wizard (Figure 5-48 on page 168).
Also, the Do not copy volume option is not available for PPRC Extended Distance.
In addition, the Suspend PPRC after established complete option is not available for PPRC
Extended Distance. The reason this requirement is not imposed is that the duplex state is not
enforced by PPRC Extended Distance.
Tip: Alternatively, you can use the Logical Subsystem panel to select Establish
Synchronous PPRC Pairs for an entire LSS. For further information, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.

174
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.11.5 Converting a PPRC-XD copy pair to Synchronous PPRC
There are two common situations when you would convert a Peer-to-Peer Remote Copy
(PPRC) copy pair from PPRC Extended Distance (PPRC-XD) mode to Synchronous PPRC:

Situation 1: You have used PPRC-XD to complete the bulk transfer of data in the creation
of many copy pairs, and you now want to convert some or all of those pairs to
Synchronous PPRC mode.

Situation 2: You have PPRC-XD copy pairs for which you want to make FlashCopy
backups on the remote site. You convert the pairs temporarily to Synchronous PPRC
mode in order to obtain a point-in-time consistent copy, as discussed in 5.11.6, “Creating a
backup copy of volumes via PPRC Extended Distance” on page 176.
For an in-depth discussion of the use of PPRC Extended Distance, see the IBM Redbook,
IBM TotalStorage Enterprise Storage Server: PPRC Extended Distance, SG24-6568.
You can convert a PPRC Extended Distance copy pair to a synchronous pair using the
Volume panel. Select the LSS within which you want to convert the PPRC Extended Distance
copy pair to a synchronous pair. Click on the primary PPRC volume. In the Target column of
the Volumes panel, the secondary Volume of the PPRC pair will be displayed (Figure 5-54).
Figure 5-54 Volumes panel: Converting PPRC - XD to Synchronous PPRC
Tip: Alternatively, you can use the Logical Subsystem panel to select Extended Distance
PPRC for an entire LSS. For further information, refer to the IBM TotalStorage Enterprise
Storage Server Web Interface User’s Guide, SC26-7448.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
175
Once you have selected the source volume, right-click the target volume, then right-click it
again to open the Task Wizard (Figure 5-55).
Figure 5-55 Task Wizard: Convert PPRC-XD to Synchronous PPRC
Select the Establish Synchronous PPRC copy pair option and click Next. On the next
panel, you can select the Copy out-of-sync cylinders only option (Figure 5-56).
Figure 5-56 Task Wizard: Establish PPRC pair copy options
Click Next. From the next panel, you can either Save, Run, or Cancel the task.
Once you run the task, all out-of-sync cylinders will be copied from the primary to the
secondary volume.

176
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.11.6 Creating a backup copy of volumes via PPRC Extended Distance
When you have used the PPRC Extended Distance (PPRC-XD) mode to transfer data, and
the distance between the local ESS and the remote ESS is close enough for you to establish
a Synchronous PPRC copy pair that can achieve the full duplex state, you can temporarily
convert from PPRC Extended Distance mode to Synchronous PPRC mode when you want to
make a consistent copy of the volumes.
This process would be useful as a back-up copy and for data migration purposes, but would
not be particularly useful for Disaster Recovery if the source data set is subject to rapid
change. For further information, refer to Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on
page 55.
Perform the following steps to obtain a backup point-in-time consistent copy of the PPRC-XD
volumes that are at the remote site:
1.Change the copy mode for the PPRC Extended Distance copy pair to Synchronous PPRC.
Select the Suspend after establish option in the Task Wizard in creating the
Synchronous PPRC. This avoids the step of creating a separate task just to suspend the
pairs. See 5.11.5, “Converting a PPRC-XD copy pair to Synchronous PPRC” on page 174.
2.Alternatively, if you did not select the Suspend after establish option, suspend the
Synchronous PPRC copy pair after it reaches the full duplex state (consistency). See
5.11.7, “Suspending a PPRC pair” on page 176.
3.Perform a FlashCopy on the secondary volumes. See 5.10.1, “Establishing a FlashCopy
pair” on page 150.
4.Resume PPRC Extended Distance mode for the copy pair. If you have saved the task that
created the original PPRC-XD relationship (see 5.11.4, “Establishing a PPRC Extended
Distance (PPRC-XD) copy pair” on page 173“), you can invoke the saved task from the
Tasks panel. For details, see 5.7.4, “Modifying a task” on page 147.
5.11.7 Suspending a PPRC pair
When you suspend a copy pair, PPRC stops transferring data to the target volume. Because
the primary ESS keeps track of all changed cylinders on the source volume, you can resume
PPRC operations at a later time. To resume operations, run a task that resynchronizes the
PPRC copy pair. See 5.11.3, “Establishing a Synchronous PPRC pair” on page 167.
ESS Copy Services now provides an alternative to creating a separate task that suspends a
PPRC pair. When you establish the pair, the Task Wizard provides the option Suspend PPRC
after Establish. For details, see 5.11.3, “Establishing a Synchronous PPRC pair” on
page 167“.
To suspend a PPRC copy pair, you use the Volumes panel. This procedure is almost identical
to the one described in 5.11.3, “Establishing a Synchronous PPRC pair” on page 167, except
that you select the Suspend PPRC copy pair option in the Select task type panel of the Task
Wizard (Figure 5-57 on page 177).
Tip: Alternatively, you can use the Logical Subsystem Panel to select convert PPRC
Extended Distance copy pairs to Synchronous PPRC. Running the tasks at the LSS level
will perform the task on several volumes. For further information, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
177
Figure 5-57 Task Wizard: Suspend PPRC copy pair
If you click Next within the next panel, you can decide whether to schedule the task with the
source volume or to schedule the task to the target volume (Figure 5-58).
Figure 5-58 Task Wizard: Specify logical subsystem to execute task panel
The task runs on either the source or target ESS. For example, if the source LSS is not
available, you can schedule a suspension of a PPRC pair with the target LSS. This is useful in
a disaster situation when the primary site has gone down.
5.11.8 Terminating a PPRC copy pair
When you terminate a PPRC copy pair, the PPRC relationship between the source and target
volume ends. The source and target volumes are removed from the PPRC configuration. You
can either use this function to terminate a Synchronous PPRC pair or an Extended Distance
PPRC pair.
Note:

If you schedule the task with the source logical subsystem, the source will become
suspended. As a result, the target will become suspended immediately after.

If you schedule the task with the target logical subsystem. The target will become
suspended. But the source might not become suspended. You may use this option only
in a disaster situation when the source LSS is not available anymore.

178
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
You can terminate a PPRC copy pair in the following ways:

By modifying a task used to establish or suspend a PPRC copy pair that has been saved
(refer to 5.7.4, “Modifying a task” on page 147 for further information)

By using the Volumes or the LSS panel
This procedure is almost identical to 5.11.3, “Establishing a Synchronous PPRC pair” on
page 167, except that you select the Terminate PPRC copy pair option in the Select task
type panel of the Task Wizard (Figure 5-59).
Figure 5-59 Task Wizard: Terminate PPRC copy pair
You will have two options when creating the terminate PPRC pairs: You can decide whether to
Schedule task with the source logical subsystem or to Schedule task with the target
logical subsystem.
5.11.9 Resynchronizing PPRC copy pairs
When you resynchronize a copy pair, all changed cylinders on the source volume are copied
to the target volume. PPRC synchronous operations are then resumed for the pair. Use the
method described in 5.11.5, “Converting a PPRC-XD copy pair to Synchronous PPRC” on
page 174 to resynchronize a PPRC pair in suspend state.
Use the Volumes panel to resynchronize a PPRC copy pair. On the left side, select the source
LSS, and on the right side, the target LSS. This is done using the drop-down menu at the top
of the Volumes menu. Select the source volume and, with a right-click, the target. Once you
have selected the source and the target, you do a second right-click on the target to bring up
the Task Wizard (Figure 5-60 on page 179).
Note:

If you schedule a

terminate PPRC pair task with the
source
logical subsystem and you
run the task successfully, the source and the target volume become in simplex state.

If you schedule a

terminate PPRC pair task with the
target
logical subsystem and the
task has been completed successfully, the source volume is in suspended state, and
the target volume is in simplex state. This option is useful in a disaster situation when
the primary site has gone down.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
179
Figure 5-60 Task Wizard: Establish Synchronous PPRC copy pair
Select the Establish Synchronous PPRC copy pair option and click Next. In the next panel,
you can select the copy options (Figure 5-61).
Figure 5-61 Task Wizard: Establish Synchronous PPRC copy pair options
Select the Copy out-of-sync cylinders only initialization option and decide whether to use
the following two options:

Permit read from secondary: Selecting this option allows host servers to read from the
PPRC secondary volume. The PPRC pair must be in a full-duplex state in order for the
host server to read the volume. Select this option if you have an operating system that
uses a Logical Volume Manager that reads data structures from the header of a
secondary volume to update internal databases such as the ODM under AIX, or
/etc/lvmtab under HP-UX. For further information, refer to Chapter 8, “Open systems
specifics” on page 407.

180
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

Suspend PPRC after establish complete: If you select the full-copy option described
above, then selecting this option means that after the data is completely copied to the
target volume, the target volume goes into the suspended state. Selecting this option if
you want to suspend the pair right after the primary and secondary volume are in sync.
Click Next. From the next panel, you can either Save, Run, or Cancel the task. Once you run
the task, all out-of-sync cylinders are copied from the primary to the secondary volume.
5.11.10 Creating a PPRC Consistency Group
Use the Paths panel to create a PPRC Consistency Group, which is a set of PPRC volume
pairs that have the same source and target LSS. When an error occurs that affects any of
these volumes, the ESS causes the volume where the error is detected to enter a suspended
state. If the volume is participating in a Consistency Group, it enters a queue full state. The
primary host temporarily queues all the updates to the primary volumes. ESS Copy Services
creates the Consistency Group when you run a task to establish a path between the primary
and secondary LSSs. For further information about PPRC Consistency Groups, refer to 4.5,
“Asynchronous PPRC” on page 74.
To create a PPRC Consistency Group, click Paths in the navigation frame of an ESS Copy
Services panel. The Paths panel opens (Figure 5-18 on page 143). From the Select Source
Subsystem list, select the logical subsystem (LSS) from which you want to create the paths.
Click the path in the Path Origin column. The ESS automatically displays the physical targets
that you can access through this connection in the Common Storage Server Targets column.
Click a physical target to select it. The ESS displays the LSS targets that you can access from
this physical target in the Logical Subsystem Targets column. Select the target storage server
that you want to select. Right-click the same target to open the Select task type panel of the
Task Wizard.
Figure 5-62 Task Wizard: Establish paths
Click Next and select the path options in the next panel (Figure 5-63 on page 181).
Tip: Alternatively, you can use the Logical Subsystem Panel to select Resynchronize
PPRC Copy Pairs. For further information, refer to the IBM TotalStorage Enterprise
Storage Server Web Interface User’s Guide, SC26-7448.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
181
Figure 5-63 Task Wizard: Select path options panel
Select PPRC consistency group and click Next. The Define task panel of the Task Wizard
opens (Figure 5-64).
Figure 5-64 Task Wizard: Define Task panel
You can either click Save to save the task and run it later, or click Run to run the task
immediately.
Attention: When you create paths for PPRC Consistency Groups, it is important to save
the task. After you run the Freeze command and corrected the problem that caused the
suspension, you need to reestablish the paths. You must reestablish the paths before you
run the task that resynchronizes the PPRC volumes.

182
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.11.11 Freezing a PPRC Consistency Group
Freezing a PPRC Consistency Group stops all operations from being propagated to the
secondary volumes. The freeze operation is best implemented by creating and saving tasks
for all the associated LSS pairs and then grouping the tasks. When you need to perform a
freeze operation, the group can be run from the Task panel or from the command-line
interface. See 5.7.1, “Grouping and ungrouping tasks” on page 145 for information on
grouping tasks.
For the freeze operation to work and create a consistent image of the data at the remote site,
the Establish Paths task must indicate that the LSS to LSS relationship is part of a
Consistency Group (see 5.11.10, “Creating a PPRC Consistency Group” on page 180).
The task is invoked when a freeze trigger event (for example, an e-mail notification, an SNMP
alert, or a host-detected failure condition) is detected. Following the freeze operation, a
consistency-group-created operation should be run to stop any long busy states that the
volumes are in. See 4.5, “Asynchronous PPRC” on page 74 for more information about the
freeze and consistency-group-created operations.
To create a task that freezes a Consistency Group, click on the Logical Subsystems button
in the navigation frame of an ESS Copy Service panel. The Logical Subsystems panel opens,
and within the Select list in the upper right corner of the panel, you select the source ESS. A
panel showing all the logical subsystems (LSSs) is displayed, with the appropriate names and
serial numbers indicated below each LSS icon (Figure 5-65 on page 183). Click on the source
LSS and click the select list again. Select the target ESS. The LSSs of this ESS are displayed
and you right-click the target LSS. The LSS ID of the target LSS turns red.
Note: You can also create a Consistency Group including all the volumes into one LSS by
clicking on the PPRC consistency group check box in the LSS properties panel (“PPRC
Consistency Group” on page 140).

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
183
Figure 5-65 Logical Subsystems panel
Right-click the target LSS again to open the Task Wizard. Select Freeze PPRC Consistency
Group (Figure 5-66).
Figure 5-66 Task Wizard: Select task type
Within the next panel, you can decide whether to Run, Save, or Cancel the task. If you want
to save the task, type the task name and the task description in the appropriate fields. Click
Run to execute the task immediately.

184
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
5.11.12 Thawing a PPRC Consistency Group
The thawing of a PPRC Consistency Group (consistency-group-created operation) follows the
successful freeze for all LSSs in the Consistency Group. It allows normal I/O operations from
the host to the ESS to resume. For further information, refer to 4.5, “Asynchronous PPRC” on
page 74).
You can either use the Task panel to modify a

Freeze PPRC Consistency Group

task to
create an appropriate task to thaw the Consistency Group (refer to 5.7.4, “Modifying a task”
on page 147 for further information), or you can use the Logical Subsystems panel to create a
task to thaw a PPRC Consistency Group.
This procedure of using the Logical Subsystems panel is almost identical to the previous
procedure, which you can use to freeze a PPRC Consistency Group, except that you select
the PPRC consistency created option in the Select task type panel of the Task Wizard
(Figure 5-66 on page 183).
5.11.13 PPRC Failover and Failback
PPRC Failover and Failback operations can be used in a disaster recovery process. The
tasks are presented in detail in Chapter 9, “Disaster Recovery” on page 453. They will be
combined to restart the applications using the volumes at a recovery site, and establish a
PPRC from the recovery site to the primary site.
We will discuss here the behavior of the two tasks, without giving too many details about the
status of the volumes. To get more information, you can refer to 6.15.11, “Performing a PPRC
Failover” on page 293 and 6.15.12, “Performing a PPRC Failback” on page 296.
PPRC Failover
This task is part of the process used to restart a production environment using the volumes
from the recovery site. See Chapter 9, “Disaster Recovery” on page 453 for details. In a
disaster recovery process, the failover task is followed by a failback task once a path from the
recovery site to the production site is established.
Use the Volumes panel to establish the PPRC pairs. The process is identical to the one used
when establishing a PPRC relationship.
When creating the PPRC Failover task you must be very careful when choosing the source
volume and the target volumes for the task:

The secondary volume of the established PPRC pair should be selected as the primary
(source) volume for the new PPRC Failover task. Click to select the volume.

The Primary of the established PPRC pair should be selected as the secondary (target)
volume for the new PPRC Failover task. Right-click to select the volume.
The failover is invoked by selecting the Failover option in the Task Wizard panel (Figure 5-67
on page 185).
Attention: After creating and saving the task to freeze a PPRC Consistency Group, you
should create the corresponding consistency-group-created task. We recommend that you
modify the task that you just created so that it runs a consistency-group-created operation,
and then name and save the task. For further information about modifying a task, refer to
5.7.4, “Modifying a task” on page 147.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
185
Figure 5-67 Task Wizard: Establish PPRC pair copy options
Once the PPRC Failover task is established on a Synchronous PPRC pair, the volumes states
change:

The initial primary volume at the production site remains unchanged. Its state is “Source
volume of a PPRC relationship”. The target volume of this relationship is the volume at the
recovery site. The status of the volume at production site is “Full duplex” of the
“Synchronous PPRC” relationship. If you perform any write operation on this volume, its
status will change from “Full duplex” to “Suspended”.

The initial secondary volume at the recovery site changed from “Target” volume to the new
state of “Source volume of PPRC relationship”. The target of this relationship is the initial
primary volume at the local site. The status of the volume at recovery site is “Suspended”
of “Synchronous PPRC” relationship.
After the PPRC Failover task is established, the volume at the recovery site is available to be
mounted and used by a host. You can use this volume to restart your application from the
recovery site.
PPRC Failback
This task is part of the process used to restart a production environment using the volumes
from the recovery site (see Chapter 9, “Disaster Recovery” on page 453, for details). The
failback task is generally performed after a failover task.
After establishing a PPRC Failover task, the volume at the recovery site is available to be
mounted and used by a host. You can use this volume to restart your application from the
recovery site (see previous section). Establishing a PPRC with the Failback option will
resynchronize the PPRC volumes at the recovery site with their peer volumes at the
production site.

186
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Use the Volumes panel to establish the PPRC pairs. The process is identical to the one used
when establishing a PPRC relationship.
When creating the PPRC Failback task, you must be very careful when choosing the source
volume and the target volumes for the task.

The primary (source) volume of your new PPRC Failback task must be the volume at the
recovery site. This volume was initially the secondary volume of the PPRC relationship.
After the PPRC Failover task established, this volume state changed to become a primary
(source) volume in a “Suspended” state. Click to select the volume.

The secondary (target) volume of your new PPRC Failback task must be the volume at the
production site. This volume is the primary volume of the initial PPRC relationship. After
the PPRC Failover task established, this volume state remained unchanged. If you
perform any write operation to this volume after the PPRC Failover task was established,
then its status changes to “suspended”. Right-click to select the volume.
The failback is invoked by selecting the Failback option in the Task Wizard window
(Figure 5-68).
Figure 5-68 Task Wizard: Establish PPRC pair copy options
Once the PPRC Failback task is established, the PPRC relationship is resynchronized
between the recovery site and the production site.

The volume at the production site is now the secondary (target) volume of the PPRC
relationship.
Important: Before you run the PPRC Failback task, it is necessary to establish a PPRC
path from the recovery site to the production site between the concerned LSSs.

Chapter 5. ESS Copy Services Web User Interface prior to LIC 2.2.0
187

The volume at the recovery site is now the primary (source) volume of the PPRC.

The status of the two volumes will be first “Copy pending” than “Full duplex” when the copy
of the out-of-sync sectors is finished.

188
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
189
Chapter 6.
ESS Copy Services Web User
Interface: LIC 2.2.0 and later
There are three different methods of managing ESS Copy Services in the open systems
environment:

A Web-based Interface

A Java-based Command Line Interface (CLI)

And, starting with LIC level 2.3.0, the ESS API has been enhanced to support Copy
Services configuration and use for PPRC and FlashCopy
It is very important to note the fact that the Web User Interface panels and icons have had
significant changes with LIC 2.2.0 and also in later levels. These changes were not only to
improve the look-and-feel for the users, but also related to the new functions that became
available: LIC 2.2.0. brought in PPRC Version 2 and FlashCopy Version 2, LIC 2.3.0. brought
in PPRC over Fibre Channel links, and LIC 2.4.0 introduced Asynchronous PPRC.
In this chapter, we explain how to use and set up the ESS Copy Services Web User Interface
(WUI) when the ESS is at the minimum LIC level 2.4.0. Therefore, this chapter describes how
to use the Web User Interface when you are using PPRC Version 2 and FlashCopy Version 2,
as well as when Fibre Channel links are used.
The Web User Interface for ESSs having LIC levels prior to 2.2.0. is discussed in Chapter 5,
“ESS Copy Services Web User Interface prior to LIC 2.2.0” on page 123.
The usage of the Command Line Interface is described in Chapter 7, “ESS Command Line
Interface” on page 377.
The usage of the ESS API is described in Appendix A, “ESS Application Programming
Interface (API)” on page 533.
6

190
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.1 Overview and requirements
ESS Copy Services runs within the IBM TotalStorage Enterprise Storage Server (ESS). Since
LIC code 2.2.0, you can define two servers in the Copy Services Domain to manage the Copy
Services in two ways: the
dual-active server
s
mode and the
mixed mode
(one active server
and one passive server). The active servers are responsible for maintaining all Copy Services
related information. One of the ESS clusters in the Copy Services Domain has to be defined
as the active server (ServerA). Optionally, you can define a second ESS cluster in the Copy
Services Domain to act as the second active server (ServerB).
On each ESS cluster that is intended to use Copy Services, there is a Copy Services client
running that communicates with the Active Copy Services server.
Refer to Chapter 2, “Implementing ESS Copy Services” on page 17 to get more information
about the Copy Services Domain and the different modes (dual-active and mixed-mode) for
these domains.
Access to Copy Services is provided through a Web browser. Using a Web browser offers the
possibility to easily control the ESS copy functionality over the network from any platform for
which the browser is supported.
A Web server running in the ESS provides a Web interface that you can use to manage the
ESS through a Web browser. The Web browser must be running on a workstation connected
to the ESS through the IBM TotalStorage Enterprise Storage Server Network (ESSNet). The
ESSNet is the ESS access facility that IBM installs when they install your ESS. The ESSNet
consists of a switch (hub) and other networking components and the dedicated IBM
TotalStorage Enterprise Storage Server Master Console (ESS Master Console).
In addition to using a Web browser on the

ESSNet Master Console to connect to your ESS,
you can also use a Web browser running on your own workstation, either by connecting your
workstation directly to the ESSNet hub, or by connecting your workstation to your intranet and
connecting your intranet to the ESSNet hub. If you use your own workstation, IBM
recommends that it have at least 128 MB of memory.
The ESS Copy Services require one of the following Internet browsers:

Netscape Navigator

Microsoft Internet Explorer (MSIE)
For supported versions of Netscape Navigator and Internet Explorer, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.
6.2 ESS Copy Services WUI
You can either use the ESS Master Console, the ESS-Net console, or a Web browser
installed on a workstation connected through the ESSNet to access the ESS.
The ESS Master Console differs from other workstations in the way you access your ESS.
The desktop of the ESS Master Console has a Netscape icon with the label ESS Specialist
Launcher. That icon accesses a panel that provides a list of the IP addresses of your ESS
clusters. Clicking on the address of the cluster that you wish to access will cause the resident
ESS Launch panel in the selected cluster to open.
If you use a workstation other than the ESS Master Console to access your ESS, you must
enter the desired cluster address in the Address field of your Web browser. This address,

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
191
which you click on in the
Specialist Launcher
of your ESS Master Console or that you type in
the address field of your Web browser, is the host name alias or the dotted decimal IP
address of one of the clusters of your ESS. IBM configures these addresses in your ESS at
installation time, based on the addresses you enter in the Communications Resources work
sheet that you submit for your ESS. For details, see the IBM TotalStorage Enterprise Storage
Server Introduction and Planning Guide, GC26-7444.
The first page to open, the ESS Launch

panel in Figure 6-1, is the same for either of the two
clusters in the ESS, so access to either cluster allows you to configure the entire machine.
The ESS Launch panel contains two buttons related to Copy Services:

The Copy Services button to connect to one active Copy Services server.

The Tools button to connect to the Copy Services Web domain configuration and the
Copy Services Help panel.
Figure 6-1 ESS welcome panel
6.3 Web Copy Services Domain: Tools and configuration
Since Version 2.2.0, the Copy Services allow the user to define the Web Copy Services
Domain using a browser. This configuration is performed using the different options accessed
via the Tools panel. Since LIC 2.3.0, the Copy Services Domain supports up to eight ESSs.
Before going further, refer to Chapter 2, “Implementing ESS Copy Services” on page 17 to
clearly understand all the concepts of the Copy Services Domain.
6.3.1 Copy Services Tools main menu
From the ESS Launch panel, click Tools. A new panel will appear with the name Tools: Copy
Services Main Service Page (Figure 6-2 on page 192).

192
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-2 Copy Services Tools panel
From this page, you will be able to perform the following operations:

Determine IPs for Copy Services active servers

Define Copy Services active servers

Define Copy Services clients

Recovering from an
Unable to Connect to the Copy Services Server
message

Restart Copy Services on this cluster

Domain-wide Reset

Disable Copy Services

Launch the Trouble Shooting and disaster recovery help
Click on the links to understand what each action will perform and to access the Available
actions menu presented in Figure 6-3 on page 193.
6.3.2 Defining and restarting the Web Copy Services Domain
A Copy Services Domain consists of two servers (ServerA and ServerB) and the clients that
are associated with those servers. To set up your domain, you must register the clusters and
clients information using the Tools panel.
The Copy Services Web domain is described in detail in Figure 6-3 on page 193.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
193
Once all the ESSs in the domain are connected into the same LAN, the actions you perform
from the Tools panel are:
1.On
each ESS
in the Copy Services Domain: Define the active servers for the Copy
Services Web domain.
2.On t
he two clusters acting as ServerA or serverB
: Define the clients in the Copy Services
Web domain.
3.On
each ESS
cluster in the domain: Start the Copy Services.
An alternative to the restart process described above is the Domain-wide Reset action.
To perform these configurations and actions, use the buttons from the Tools panel (see
Figure 6-3).
Figure 6-3 Copy Services action buttons from the Available Actions panel
Defining the active servers
Starting with LIC 2.2.0, dual-active servers for Copy Services in the same domain are
supported. The two servers are designated as ServerA and ServerB. Depending on your
domain configuration, the two servers will act in dual-active mode or in mix-mode. See
Chapter 2, “Implementing ESS Copy Services” on page 17.
The definitions of the two servers is performed using the Tools panel on each ESS in the
Copy Services Domain.You need to perform this definition for all of the ESSs in the Copy
Services Domain. Once the definition is submitted on one cluster (using the “Submit
Configuration” button in Figure 6-4 on page 194), the configuration is automatically updated
to the alternate cluster on the ESS.
You define the servers by entering the IP addresses of the two clusters in the Define Copy
Services Active servers panel. This panel is accessed by clicking on the Define servers
button. An IP address for serverA is mandatory; defining ServerB is optional. If you do not
define ServerB, then you will not have dual-active mode for the Copy Services, since only one
server will be active for the Copy Services.

194
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-4 Define Copy Services Active Servers
Defining the clients
After you define the Active servers, the next step consists of defining the list of all the ESSs in
the Copy Services Domain. With LIC 2.3.0 installed, up to eight ESSs can be defined in a
Domain. The two active servers (serverA and serverB) must appear in this list.
You will define the Copy Service clients on the two clusters defined as active servers. Defining
the clients on ServerB is not mandatory but highly recommended. If you do not define the
clients on ServerB, you cannot use the Domain-wide Reset action.
Define the Copy Services clients by clicking the Define Clients button. Input the information
requested for each of the client clusters (see Figure 6-5 on page 195).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
195
Figure 6-5 Defining the clients
Once all the clients are defined, click on the Submit Configuration button, as in Figure 6-6,
to update the active server with the configuration.
Figure 6-6 Define clients button
Start/Restart Copy Services
This operation is mandatory after you define the active servers and the clients to be able to
use the Copy Services options.
To restart Copy Services, click the Tools button in the ESS welcome panel to open the Tools
panel. On the Tools panel, click on the Recover from an ‘Unable to Connect to the Copy
Services Server’ message to link to the restart procedure. Be sure you understand the
result of implementing a restart before proceeding. When you use this function, be aware
that:

You will lose any PPRC or FlashCopy tasks that have not completed.

Established PPRC and FlashCopy relationships are maintained.

You cannot submit any additional tasks through the CLI until ESS Copy Services has
re-initialized.
page can be
scrolled down
for up to
8 Clients

196
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

You must exit the Copy Services panel and re-enter it to receive valid updates from the
Copy Services server.
If you decide to restart ESS Copy Services, click the Reset CopyServices button. A warning
message,
Resetting the ESS Copy Services
, appears. Click on OK to perform the reset.
Domain-wide Reset
You can use the Domain-wide Reset button to reset each cluster within the Copy Services
Domain with the current active server.
To use the Domain-wide Reset function, you must have a common Web Administrator level
user ID and password defined on all ESS clients in the ESS Copy Service Server domain.
Copy Services clients must be configured on the ESS clusters that are defined as Server A
and Server B to support the Domain-wide Reset function.
To perform a Domain-wide Reset, click the Tools button in the ESS welcome panel to open
the Tools panel. On the Tools panel, click on the Perform a Domain-wide Reset button to link
to the Domain-wide Reset procedure. Be sure you understand the result of implementing a
Domain-wide Reset before proceeding. When you use this function, be aware that:

You will lose any PPRC or FlashCopy tasks that have not completed.

Established PPRC and FlashCopy relationships are maintained.

You cannot submit any additional tasks through the CLI until ESS Copy Services has
reinitialized.

You must exit the Copy Services panel and re-enter it to receive valid updates from the
Copy Services server.

This function will not be successful on the cluster that does not have the Domain-wide
Reset code installed.

For the clusters that do not have the Domain-wide Reset code, the reset has to be
performed on the each cluster separately.
If you decide to restart ESS Copy Services, click the Domain-wide Reset button. A warning
message,
Resetting the ESS Copy Services
, appears. Click on OK to perform the reset.
6.4 Connecting to the Copy Services Web User Interface
When you click on the Copy Services button in the Welcome panel (Figure 6-1 on page 191),
a new panel will prompt you to select which active server you want to connect to (Figure 6-7
on page 197).
By clicking Start with serverA or Start with serverB, you will open the Copy Services Web
User Interface in another panel. If you have not previously selected one of those two buttons,
Attention: Use this function only if you are certain that no other recovery options are
available.
Note: Only ESSs that are running LIC level 2.2.0 or higher will be reset
Attention: Use this function only if you are certain that no other recovery options are
available.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
197
you will be prompted for the user name and the password before starting the Copy Services
Web User Interface.
Figure 6-7 Start CopyServices panel
The message panel shown in Figure 6-8 will be displayed while connecting to the Copy
Services Server.
Figure 6-8 Connecting to the Copy Services server
Once the connection to the Copy Services server is successful, the main menu of the
ESS Copy Services

WUI will be displayed (Figure 6-9 on page 198). From here, you can
access all Copy Services menus by selecting one of the buttons on the left side.
Attention: Once the Copy Services Web User Interface is started, do not close the Start
Copy Services panel, or it will also close the Copy Services GUI panel.

198
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-9 Copy Services GUI: Welcome panel
There are six buttons in the Navigation panel on the left. The actions and panels available via
these buttons are discussed in the sections below:

Introduction button: This button brings you the initial Copy Services panel (Figure 6-9).

Volumes button: Refer to 6.5, “Volumes panel” on page 199.

Logical Subsystems button: Refer to 6.6, “Logical Subsystems panel” on page 207.

Paths button: Refer to 6.7, “The Paths panel” on page 213.

Tasks button: Refer to 6.8, “Tasks panel” on page 217.

Administration button: Refer to 6.9, “Administration panel” on page 222.

Exit button: Refer to 6.10, “Exiting ESS Copy services” on page 224.
6.4.1 Failure to connect to the ESS Copy Services
If you receive the following message when you click Copy Services in the navigation frame of
the ESS Launch page:
Failed to connect to Copy Services Server. Server may be down or not configured and cannot
access ESS Copy Services.
the failure is most likely the result of at least one of the following three conditions:

The Copy Services active servers are not defined. (You need at least one active server
defined).
– If you have successfully accessed ESS Copy Services before, this would not be the
source of the problem.
– If this is the first time that you are connecting to ESS Copy Services, see 6.3.2,
“Defining and restarting the Web Copy Services Domain” on page 192.

Check for the active server configuration and verify that the Copy Services are running on
the servers. If the Copy Services server is not running, see “5.3.3, “Restarting ESS Copy
Services” on page 128 for instructions.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
199

Your Web browser has a temporary operational or connection problem.
For Web browser problems, take these corrective actions:
1.Close any open ESS Copy Services browser panel.
2.Access ESS Copy Services again by clicking Copy Services.
3.If step 2 is unsuccessful, close all browser windows (if you have ESS Specialist running,
first complete any unfinished task definition), then restart your browser.
4.Again, access the ESS Launch page and click Copy Services.
5.Make sure you have IP connectivity to the ESS cluster with the Server you are trying to
connect to. When you select ServerA or ServerB, you are redirected to that IP address.
6.If Copy Services still does not launch, consider restarting the workstation that is running
your browser.
6.4.2 Restart the ESS Copy Services
If ESS Copy Services is not available, and you fail to connect to any of the active servers, the
tools panel proposes two ways to restart ESS Copy Services:

Reset CopyServices procedure

Domain-wide Reset procedure
Be aware that you need to have administration authority to do this. Refer to “Start/Restart
Copy Services” on page 195 and “Domain-wide Reset” on page 196.
6.5 Volumes panel
Volumes are defined with the ESS Specialist to provide a fixed storage capacity to the
connected host system. They are the base components of each data copy task. The ESS
assigns each volume a unique eight digit identifier (ID). This identifier is used to address each
volume within the ESS.
From the Volumes menu, you will be able to:

Find a volume.

Display volumes based on a filter.

View information about a volume.

Establish and withdraw FlashCopy pairs with different options.

Establish, suspend, and terminate Synchronous PPRC copy pairs with different options.

Establish, suspend, and terminate PPRC Extended Distance copy pairs.

Convert a PPRC Extended Distance copy pair to Synchronous PPRC.

Establish multiple selection mode to select volume-pairs when creating relationships for
FlashCopy and PPRC.

Establish an Asynchronous Cascading PPRC relationship.
Note: For all three conditions noted above, if you have the CLI active on one of your host
systems, you can verify that there is a connection to the active Copy Services server with
the
rsTestConnection
command. For details, see Chapter 7, “ESS Command Line
Interface” on page 377.

200
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The window is divided in two panels (see Figure 6-10). In each panel you will display all the
volumes in the same LSS.
Figure 6-10 Volume panel with source and target selected

Use the left panel to display and select the volumes you will use as source volumes for
FlashCopy or as primary volumes for PPRC.

You will use the right panel to display and select the volumes you will use as target
volumes for FlashCopy or as secondary volumes for PPRC.
To display the volumes shown in Figure 6-10, do the following:
1.Select the source LSS using the scrolling list above the panel on the left.
2.Select the target LSS using the scrolling list above the right panel.
The source and target logical subsystem are specified in the following way: Device type (4
digits):ESS Serial number (5 digits):LSS number (2 digits). For example, 2105:22331:16
designates the LSS number 16 on the ESS, which has the serial number 22331.
The Command Line user interface provides a very useful command that will allow you to
display all the volumes assigned to a host in the panels. This command is
rsPrimeServer
.
1.Issue the
rsPrimeServer
command from any host. A list of volumes is sent by the host to
the Copy Services Server. Refer to Chapter 7, “ESS Command Line Interface” on
page 377.
2.In the Volumes panel, select the host name in the source and target list to display all the
volumes associated with this host (see Figure 6-11 on page 201).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
201
Figure 6-11 Example of results of the rsPrimeServer command
6.5.1 Volumes panel icons
The Volumes menu shows all volumes defined within one LSS. To the right of each volume
icon, you will find its unique serial number. To the right of the serial number, you will find one
or more icons. Those icons will describe how the volume is being used in any Copy Services
relationship. No icon means that the volume is not in use by any copy relationship.
In the Volumes panel, you get a short description about the meaning of each icon when you
click on the Legend button located at the upper right corner of the panel.
Table 6-1 presents the meaning of the different icons and colors in the Volumes panel.
Table 6-1 Meaning of the icons and colors in the Volume panel
Tip: Additionally, using the CLI command
esscli set volume
, you can add a label on the
the volumes to help you identify volume functions. This label will appear below the volume
serial number.
Icon Description
Volume icon. This icon designates a volume created in the ESS. This icon is always
displayed whether this volume is or is not in a copy relationship.
Designated source volume. When you right-click on a volume to select it as a source, then
the volume icon and its serial number are surrounded by a rectangle filled with blue color.
Designated target volume. When you click on a volume to select it as a target, the volume
icon and its serial number are surrounded by a rectangle filled with red color.

202
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Designated source and target volume. When you left-click, followed by a right-click, on the
same volume, then the volume icon and its serial number are surrounded by a rectangle
filled with grey color.
PPRC source volume. The volume is a source volume in an established PPRC
relationship. The volume is in duplex mode for this relationship.
PPRC target volume. The volume is a target volume in an established PPRC relationship.
The volume is in duplex mode for this relationship.
PPRC source volume copy in progress. The volume is a source volume in an established
PPRC relationship. The volume is in duplex pending mode for this relationship.
PPRC target volume copy in progress. The volume is a target volume in an established
PPRC relationship. The volume is in duplex pending mode for this relationship.
PPRC source suspended. The volume is a source volume in an established PPRC
relationship. The volume is in suspended mode for this relationship.
PPRC target suspended. The volume is a target volume in an established PPRC
relationship. The volume is in suspended mode for this relationship.
PPRC Extended Distance source. The volume is a source volume in an established
PPRC-XD relationship.
PPRC Extended Distance target. The volume is a target volume in an established
PPRC-XD relationship.
PPRC Trusted Primed for Resynchronization.
FlashCopy source. The volume is a source volume in an established FlashCopy
relationship.
FlashCopy target. The volume is a target volume in an established FlashCopy relationship.
Change recording enabled. This icon displays on the source and target volume when you
establish a FlashCopy with the Change recording enabled option. The pair can be used to
perform Incremental FlashCopy or Reverse Restore FlashCopy.
Icon Description

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
203
6.5.2 Volume Information Panel button
You can get detailed information about a single volume by selecting the volume and clicking
on the Information Panel button.
You can leave the Information Panel open as you work in the volume panel and select
different volumes.
Figure 6-12 on page 204 shows the information panel of a volume that is the source of two
FlashCopy relationships.
Dataset FlashCopy (for S/390 only).
Volume Copy. The volume is the target of a FlashCopy relationship. This icon remains even
if the FlashCopy relationship is withdrawn. To remove this icon manually, you need to
withdraw the relationship with a Withdraw from target command.
Volume Session Active. The volume is active in a session.
Volume Session Join Pending. The volume is joining the session, but is not yet active.
Volume Session Remove Pending. This volume is in the process of being removed from
the session; when complete, this icon will be removed.
Multiple FlashCopy relationship. The volume is a source volume for more than one
FlashCopy relationship.
The state of this volume cannot be determined.
Icon Description

204
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-12 Volume Information Panel
The following information is displayed for a volume:

The storage server (ESS) serial number.

Logical subsystem (LSS) designations.

The volume number.

The volume serial number.

The LSS type (“S/390” or “Open System”).

The volume type (Fixed Block for open systems; 9337, 3390, and other types for S/390).

The volume capacity in sectors (for fixed block) or in cylinders (CKD format).

PPRC status: If the volume is in an active PPRC relationship, it indicates whether the
volume is a primary (source) volume or a secondary (target) volume. The properties for
the relationship, the volume status, and the number of out-of-synchronous sectors are
displayed in the Full Volume PPRC peers (Figure 6-12). If there are no active PPRC
relationships active on a volume, a value of
None
is displayed.

FlashCopy Status: If the volume is in any FlashCopy active relationship, it indicates
whether the volume is a source or a target. Information about the corresponding volumes
and out-of-synchronous sectors are displayed in the Full Volume FlashCopy Peer
(Figure 6-12). If there are no active FlashCopy relationships active on a volume, a value of
None
is displayed.

Any active extended remote copy (XRC) operations (this function is available for S/390
only).

Whether the Concurrent Copy function is active (this function is available for S/390 only).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
205

The status of Asynchronous PPRC for this volume. If it is not in an active session, then the
status will be Volume not in Session.
6.5.3 Finding volumes
The Find button in the volumes panel enables you to search for a specific volume. The
volume must be specified by its eight digit ID.
Figure 6-13 Find volume
If the volume ID is valid, the volume is displayed in the area you selected by clicking on
Display in source area or Display in target area, as in Figure 6-13.
The volume is displayed on the top of the area and is automatically selected as a
source

volume (as if you selected the volume and performed a right-click on it).
6.5.4 Filtering volumes
You can filter the volumes displayed in the Volumes panel. To do this, click on the Filter On
button, and select the filter volume option, as in Figure 6-14 on page 206.

206
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-14 Filter volumes panel
6.5.5 Multiple Selection Mode
When you enter in the Multiple Selection Mode, you can select multiple volume pairs when
you create tasks for both FlashCopy and PPRC. This mode is very useful for saving time
when creating the tasks.
You might use this mode when selecting the volumes for FlashCopy and PPRC tasks (for
further information, refer to 6.11.1, “Establishing a FlashCopy pair” on page 226, 6.14,
“Establishing a Synchronous PPRC pair” on page 266, and 6.15.1, “Establish a PPRC
Extended Distance copy pair” on page 271). Rather than defining one copy pair at a time, you
can define all the copy pairs at one time by selecting multiple sources and their targets in the
same task definition (Figure 6-15 on page 207).
To enter the Multiple Selection Mode:
1.Click on the Multiple Selection Mode button at the bottom of the panel. This allows
multiple pairs to be defined.
2.Select the desired source volume and right-click on the desired target volume.
3.Repeat the selection of source and target volumes until you have selected all volume
pairs.
4.Right-click one of the target volumes again and the Task Wizard opens. You can decide
whether to run one of the following operations:
– Establish Synchronous PPRC copy pairs
– Establish PPRC Extended Distance copy pairs
– Suspend PPRC copy pairs

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
207
– Terminate FlashCopy pairs
– Establish FlashCopy pairs
– Withdraw FlashCopy pairs
To exit the Multiple Selection Mode, click the Exit Multiple Selection Mode button at the
bottom of the Volumes panel.
Figure 6-15 Multiple Selection Mode
6.6 Logical Subsystems panel
The Logical Subsystems panel displays the Enterprise Storage Servers and the logical
subsystems (LSSs) within the storage network. The storage network includes all Enterprise
Storage Servers that are configured to use the same Copy Services Server. Each of the
logical subsystems is specified by the serial number of the ESS it belongs to and its 2-digit
LSS number within the ESS.
With the Logical Subsystems panel, you will be able to:

Establish PPRC copy pairs for LSSs.

Convert PPRC Extended Distance copy pairs to synchronous pairs
for all the volumes in a source LSS and target LSS.
Restriction: You can only perform the multiple selections on volumes within the same LSS
(all sources must be in the same LSS and all targets must be in the same LSS).
Attention: If you click Exit Multiple Selection Mode before you perform any operations on
them, you lose all your previous selections.

208
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

Filter a list of LSSs.

Find an LSS.

Freeze a PPRC consistency group.

Modify the PPRC consistency group time-out value.

Remove orphaned paths.

Resynchronize PPRC copy pairs for LSSs.

Run a Consistency-Group-Created operation.

Suspend PPRC copy pairs for LSSs.

Terminate PPRC copy pairs for LSSs.

View information about an LSS.
In Figure 6-16, you can see the Logical Subsystems panel for a selected ESS. The color
indicates the state of the LSS, whether it contains volumes that are currently in a copy
relationship (source, target, or mixed), or that they are not part of a copy pair at all.
Figure 6-16 Logical Subsystem panel
Table 6-2 on page 209 explains the meanings of the icons that appear on the Logical
Subsystems panel. The Volumes panel displays similar icons for individual volumes (see
Table 6-1 on page 201).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
209
Table 6-2 Meaning of icons on Logical Subsystems panel
LSS Icon Appearance Meaning
Gray solid (
Simplex state
) All the volumes in the LSS are in simplex
state, meaning that they are not in a copy
relationship.
As above with yellow top;
indicates that this LSS is also a
Master
Blue solid (PPRC source) All the volumes in the LSS are in PPRC pairs
(Synchronous or Extended Distance) in a
non-suspended state.
As above with yellow top;
indicates that this LSS is also a
Master
Red solid (PPRC target)
As above with yellow top;
indicates that this LSS is also a
Master
Blue and white stripes (PPRC
suspended source)
All the volumes in the LSS are in PPRC pairs
(synchronous or Extended Distance) in a
suspended state.
As above with yellow top;
indicates that this LSS is also a
Master
Red and white stripes (PPRC
suspended target)
As above with yellow top;
indicates that this LSS is also a
Master

210
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.6.1 LSS Information Panel button
You can get detailed information about a single LSS by selecting the LSS and clicking on the
Information Panel button.
In our example in Figure 6-17, we selected LSS10 and opened the information panel. LSS 10
was colored as a yellow solid. In the information panel, we read that LSS10 contains six
volumes. All six volumes are currently source volumes of FlashCopy relationships.
Figure 6-17 LSS Information Panel
Yellow solid Not all volumes in the LSS are in the same
type of Copy Services relationship (this apply
for PPRC and FlashCopy relationships), and
none of the volumes are in a suspended
state.
As above with yellow top;
indicates that this LSS is also a
Master
Yellow and white stripes (mixed
types and states)
The volumes in the LSS are either the same
type, but some are in a non-suspended state,
or not all of the volumes are the same type,
but some are in a suspended state.
As above with yellow top;
indicates that this LSS is also a
Master
LSS Icon Appearance Meaning

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
211
6.6.2 Finding LSSs
You can search for a specific LSS by clicking on the Find button. The LSS must be referred to
using the following syntax: ESS serial number:LSS number.
In our example in Figure 6-18, we want to find LSS 10 in the ESS with the serial number
22331.
Figure 6-18 Find Storage Server LSS
6.6.3 Filtering LSSs
You can have many filtering options from the Logical Subsystems panel. Click on the Filter
button, and select the Filter devices volume option (see Figure 6-19).
Figure 6-19 Filtering LSS

212
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.6.4 LSS properties
To view or modify the properties of an LSS, select the LSS and click on the Properties button,
as in Figure 6-20.
Figure 6-20 LSS properties
PPRC Critical Heavy mode
This option is not available for open systems’ LSSs.
PPRC Consistency Group
To maintain consistency of data across volumes at the recovery site, volumes in a PPRC
relationship can be collected into a
Consistency Group
. If you mark this check box, then all
the volumes in that LSS will be considered as part of one Consistency Group. Creating a
Consistency Group of volumes provides the ability to temporarily queue subsequent write
operations to all PPRC consistency group volumes on a single LSS pairing when an error
occurs on one of the volumes in the group (primary or secondary), or when a total link failure
is detected between the primary and secondary LSS pair. The other way to establish a
Consistency Group is to select the PPRC consistency group option when establishing a
path between two LSSs. For further information about PPRC Consistency Groups, refer to
4.5, “Asynchronous PPRC” on page 74.
XRC session timeout
This parameter does not apply to open systems.
CONCOPY session timeout
This parameter does not apply to open systems.
Consistency Group timeout
This parameter, used by FlashCopy V2 and PPRC V2, indicates the amount of time that an
I/O is withheld from updating a primary volume of a Consistency Group in case of an error
event. This timeout enables automation software to detect that an error has occurred and to
issue commands to
freeze
all other members of the Consistency Group. When the
Consistency Group is created, this parameter is set to a default of two minutes. For further
information about PPRC Consistency Groups, see 4.5, “Asynchronous PPRC” on page 74.
For further information about FlashCopy Consistency Groups, see 3.3.4, “FlashCopy
Consistency Groups” on page 39.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
213
6.7 The Paths panel
A path is used to send data between the primary and secondary volumes of PPRC pairs.
The physical path consists of the ESCON or Fibre Channel connection between two
Enterprise Storage Servers, while a logical path describes the connection of the PPRC
source and targets.
Be aware that before you can establish PPRC copy pair relationships, you must use the Paths
panel to establish logical paths between logical subsystems for the source and the target
LSS. After you establish the paths, the Paths panel displays the current status of the paths for
the selected LSS. There can be multiple logical paths established over a single physical path.
For further information, refer to 4.9.3, “PPRC paths definition” on page 106.
From the Paths panel (see Figure 6-21), you will be able to:

Establish paths

Add paths

Remove a group of established paths

Remove one or more paths from a group of established paths

Create a PPRC consistency group

View information about paths
To view configured paths over Fibre Channel links (ESS serial number 23953, in our example)
on the Paths panel, follow the steps listed:
1.Start by clicking on View Path Status, as shown in Figure 6-21.
2.From the drop-down box, select the source ESS.
3.Click on the source LSS.
4.From the drop-down box, select the FCP.
For paths successfully configured, three blue asterisks will show under the link.
Figure 6-21 View FCP paths status
1
2
3
4

214
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To view configured paths over the ESCON links, on the Paths panel, repeat steps one through
three, but for step 4, from the drop down box, select the adapter type ESCON, as shown in
Figure 6-22.
Figure 6-22 View ESCON paths status
6.7.1 Icons and symbols in the Paths panel
Table 6-3 gives you an overview of the connection symbols used in the
Local Port / Remote

port
column on the right side of the panel.
Table 6-3 Meaning of the Paths panel icons
4
Connection icon Description
ESCON Host adapter port, with the system adapter identification
number (SAID) provided under it
ESCON director
Host server

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
215
If there are logical paths defined on an ESCON or FCP adapter, you will find three blue
asterisks directly below the adapter in the Local/Remote Ports column on the Paths panel.
Three red asterisks below the path-connection symbol mean that the last attempt to establish
the path failed.
Figure 6-23 shows an ESCON adapter SAID without and then with the path successfully
defined, and finally with a failure to establish the path.
Figure 6-23 ESCON link: No path defined, path defined, and path definition failed
6.7.2 ESCON System Adapter ID (SAID)
The ESCON adapters are specified by their System Adapter ID (SAID). Figure 6-24 on
page 216 shows the SAID of all ESS ESCON adapters.
The System Adapter IDs (SAID) are further discussed in Appendix D, “System Adapter ID
(SAID)” on page 555.
Storage server
Unknown device
Local FCP host adapter port, with the system adapter
identification number (SAID) provided under it
Remote FCP host adapter port, with serial number provided
under it
Connection icon Description

216
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-24 SAID numbers of the ESCON adapters
6.7.3 FCP System Adapter ID (SAID)
The FCP adapters are also specified by their System Adapter ID (SAID). Figure 6-25 shows
the SAID of all the Fibre Channel ports.
Figure 6-25 SAID numbers of the FCP adapters
The System Adapter IDs (SAID) are further discussed in Appendix D, “System Adapter ID
(SAID)” on page 555.
6.7.4 Display Connection Paths
If you have two ESSs that are directly connected (that is, no switch between them), you can
click Display Direct Connect Paths to display those paths.
SAID 0000
SAID 0001
SAID 0004
SAID 0005
SAID 0008
SAID 0009
SAID 000C
SAID 000D
SAID 0020
SAID 0021
SAID 0024
SAID 0025
SAID 0028
SAID 0029
SAID 002C
SAID 002D
SAID 0080
SAID 0081
SAID 0084
SAID 0085
SAID 0088
SAID 0089
SAID 008C
SAID 008D
SAID 00A0
SAID 00A1
SAID 00A4
SAID 00A5
SAID 00A8
SAID 00A9
SAID 00AC
SAID 00AD
BAY1 BAY2 BAY3 BAY4
SAID numbers of the ESS ESCON adapters
Host adapter bays viewed from the frontside of the Enterprise Storage Server
SAID 0000
SAID 0004
SAID 0008
SAID 000C
SAID 0020
SAID 0024
SAID 0028
SAID 002C
SAID 0080
SAID 0084
SAID 0088
SAID 008C
SAID 00A0
SAID 00A4
SAID 00A8
SAID 00AC
BAY2
BAY3
BAY4
BAY1
Fibre Channel
ports
Note: You must perform a refresh function for the following conditions:

If this is the very first time anyone is accessing the ESS Copy Services WUI.

After an update of the ESS microcode.

After a concurrent service action.

If the cabling for the ESCON links is modified.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
217
6.7.5 Path Information Panel button
Once an ESCON or FCP adapter is selected, you can get more information about the paths
by clicking the Information Panel button at the bottom of the Paths panel, as in Figure 6-26.
Figure 6-26 Path Information Panel
The example shown in Figure 6-26 shows a path defined between source LSS 16 on ESS
18767 and target LSS 13 on ESS 22331 using the ESCON adapters SAID0081 on the source
ESS and remote SAID0080 on the target ESS.
6.8 Tasks panel
With the ESS Copy Services, you have the possibility to save the setup of any data copy
action within a Task. This could be any kind of FlashCopy, PPRC, or path operation.
From the Tasks panel, you will be able to:

Group tasks

Modify a task

Remove a task group

Remove a saved task

Run a saved task

Ungroup tasks

View error information about a failed task

View information about a saved task
Figure 6-27 on page 218 shows the Tasks panel of the ESS Copy Services. For each task,
the name, a description, and the last status of the execution is displayed.
You will be prompted to choose a task name when you create a task. Use the following
guidelines:

The name can only contain alphanumeric characters.

The name cannot contain blanks.

The name can contain the underscore (_) or hyphen (-) characters.

218
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

The name cannot exceed 16 characters.
Figure 6-27 Tasks panel
6.8.1 Display task information
You can see information about a task by selecting the task and clicking on Information
Panel.
In our example in Figure 6-28 on page 219, we show a task that will perform a FlashCopy with
the No Background copy and Freeze FlashCopy Consistency Group options selected.
The FlashCopy will be performed on multiple volumes (there are four source volumes and
four target volumes). The task was created using the Multiple Selection Mode.
Note: In Appendix E, “Sample naming convention” on page 561, we present the task
naming convention used in this chapter. You can refer to this appendix to find some hints
and tips about naming conventions.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
219
Figure 6-28 Task Information panel
6.8.2 Grouping and ungrouping tasks
To create a group task, click the single tasks you want to group together while holding the
Shift key or the Ctrl key (mark single tasks). Once you are finished, click the Group button
and specify the group name. It is not possible to include a group into another task group. All
tasks within a task group will be processed in parallel.
Figure 6-29 Tasks panel: Grouping tasks

220
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
An example for the usage of a task group would be multiple FlashCopy pairs from different
logical subsystems or different ESSs that need to be issued all at the same time with the
Freeze FlashCopy Consistency Group option in order to do a backup using the target
volumes Consistency Group.
You can get detailed information about the setup of the task. Select the task and click the
Information Panel button at the lower right.
In our example in Figure 6-30, we show the information panel for a grouped task. This group
contains three single tasks. The three tasks are FlashCopy tasks with No copy and Freeze
FlashCopy Consistency Group options. They will be performed in two different ESSs.
Grouping tasks will cause the individual tasks to collapse into the overall group task.
Figure 6-30 Information panel: Task group
To ungroup a task, select the task and click the Ungroup button at the bottom of the Tasks
panel. It is necessary to ungroup the overall group task in order to gain access and to execute
the individual tasks.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
221
6.8.3 Removing a task
To remove a task, select the task and click the Remove button at the bottom of the Tasks
panel.
6.8.4 Running a task and viewing an error about a failed task
To run a saved task, select the task, and click the Run button at the bottom of the Tasks
panel. The task will be executed immediately.
If the task fails, a failure report is available through the Information Panel (Figure 6-31). The
failure report contains the error messages and the sense data. The messages are
documented in the IBM TotalStorage Enterprise Storage Server Web Interface User’s Guide,
SC26-7448.
You can also view the logs in the administration panel.
Figure 6-31 Information panel: Failing task report
6.8.5 Modifying a task
Use the Tasks panel to modify a previously saved task. You can change the task goals, task
options, task name, and description.
You will be prompted if you want to run, to replace the selected task, or create a new task.
You can change a task from one that establishes a PPRC copy pair to one that suspends,
re-synchronizes, or terminates a PPRC copy pair. For FlashCopy, you can change a task that
establishes a FlashCopy pair to one that withdraws a FlashCopy pair, or you can change the
options of one task.
Note: Grouped tasks cannot be modified. The tasks must be ungrouped and then the
tasks can be modified.

222
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.9 Administration panel
Use the Administration panel (see Figure 6-32) to manage the server logs, reports, and ESS
Copy Services Command Line Interface (CLI) user IDs (user IDs) and passwords. You can
also use this panel to refresh the volume and LSS information for an ESS.
Figure 6-32 Copy Services Administration panel
You can perform the following actions from the Administration panel:

Clear the server logs.

Manage the CLI user ID and password for an open systems host:
– Define the user ID and password.
– Remove the user ID and password.

Refresh the path and volume information for an ESS.

Send the ESS Network Configuration report to an e-mail address.

Send the ESS Resource Configuration report to an e-mail address.

Send the server logs to an e-mail address.

Specify the e-mail address that receives server logs, ESS Network Configuration reports,
and ESS Resource Configuration reports.

View the ESS Network Configuration report.

View the ESS Resource Configuration report.

View the server logs.
6.9.1 Working with the Copy Services logs and reports

The Copy Services server maintains the following logs and reports:
– Copy Services status log (copyservices.log): This contains messages that are
associated with user actions issued through ESS Copy Services. This log includes
messages associated with such actions as saving tasks and executing tasks. The log

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
223
also includes information about the success or failure of the tasks. You can click the
error message
to get a description of the problem and its possible solutions.
– Copy Services Timing log (rsCStiming.log): This contains information about the
time it took for data-copy functions (PPRC or FlashCopy) to complete on a specific
volume.
– ESS Network Configuration report: The ESS Network Configuration report contains
important information about your network-defined ESS resources:
• IP addresses for your primary and Backup Copy Services servers
• IP addresses and host names for the ESS clusters
• IP addresses for the ESS clients
– ESS Resource Configuration report: The ESS Resource Configuration report
contains the following important information about your ESS resources:
• The 2105 connection information (SAIDs, port types, and values)
• PPRC path information (SAIDs, LSSs, remote SAIDs, ports, and status)
• S/390 or zSeries volumes (LSSs, labels, Concurrent Copy status, FlashCopy
status, PPRC status, or XRC status)
• Open-systems volumes information (LSSs, volume serial numbers, user defined
label, FlashCopy status, or PPRC Status)
An extract of the ESS Resource Configuration report is presented in Figure 6-33.
Figure 6-33 Copy Services Administration panel
Attention: The status log is useful for problem determination. Therefore, you should
only clear it when no problems exist or when you have completed data collection for
all failures.

224
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To clear a log, select the log in the Log/Report list and click the Clear button. Notice that you
cannot clear a report. You can view the log or report by selecting the appropriate log/report
and click the View button.
To send a report or a log to an e-mail address, select the appropriate log/report in the
Log/Report list and click the E-mail button. Notice that you have to specify the e-mail
address in the E-mail field.
6.9.2 Refreshing path and volume information on an ESS
Use the refresh function of ESS Copy Services to get the most up-to-date information about
paths and volumes for an ESS. To perform this action, select the ESS for which you want this
information in the ESS list and click the Refresh button.
A status message opens on the top of the Administration panel noting that:
A refresh request was sent to <the selected ESS>. It may take a few minutes before the
server can refresh the data.
6.9.3 Managing the CLI user ID and password for an open systems host
Use the Administration panel to manage the user IDs and passwords for the Copy Services
Command Line Interface (CLI). Open-systems hosts that issue CLI commands to a Copy
Services server must provide a valid user ID and password as part of the command string. If
the Password Protection option is enabled and the hosts do not provide the user ID and
password, the command fails.
To enable the password protection for access to an ESS through the CLI, click Enabled in the
Password Protection field. To add users, click the Users button and the Authorized Users
dialog box will appear. Click the Add button and enter the user name and password. If you
have entered the user name and the password, click the Add button, and the new user will be
authorized. To remove a user, select the user in the Authorized Users dialog box and click the
Remove button. After you have performed all actions, click Done and the Authorized Users
dialog box will be closed.
6.10 Exiting ESS Copy services
There are several legitimate ways to quit the ESS Copy Services Web User Interface:

You can click the X at the top right corner of the ESS Copy Services panel.

You can click Exit in the navigation frame of any of the ESS Copy Services panels.

You can shut down the whole browser from another browser window.
In addition, if you simply want to return to the ESS Launch page without closing ESS Copy
Services, such as to launch ESS Specialist, you can keep the ESS Copy Services browser
window open and use the standard navigation methods for your operating system and
browser, such as Alt-Tab to switch windows.
The difference between the first two options above is nothing more than a warning message.
If you click Exit in the navigation frame of any of the ESS Copy Services panels, the warning
Attention: In a dual-active Copy Services server configuration, when you define the user
ID and password on one active ServerA, ServerA does not send them to the other Copy
Services server (ServerB) as it does for Copy Services tasks. Therefore, you must define
the user ID and password at the Copy Services ServerB in a separate step.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
225
message opens on top of the browser window and tells you that the tasks that you have
submitted will continue running after you close the window. Click OK. The ESS Copy Services
browser window closes and the ESS Launch page comes into focus. In either case, the next
time you click Copy Services on the ESS Launch panel, the ESS Copy Services browser
window opens without requiring you to log in. The browser loads a cached version of the
applet.
6.11 Performing FlashCopy operations with the WUI
In this section, we explain how to set up FlashCopy relationships between volumes in an ESS
using the Web User Interface (WUI).
Be aware of these requirements of the FlashCopy functionality:

The source and target volume must be in the same ESS. Within the same ESS, they can
be in a different LSS.

The target volume must be the same size as the source volume or larger.

A target volume can be in only one FlashCopy relationship at a time.

A Source volume can be in multiple relationships at a time. Among the multiple FlashCopy
relationships using the same volume as source volume, only one can be an
Incremental

FlashCopy relationship.

For LIC 2.4.0 or higher, with FlashCopy V2, the target can be a PPRC primary volume.
There are two different ways of establishing a FlashCopy pair:

From the Volumes panel

From the Tasks panel (once a task for a FlashCopy is created and saved)
Note: Loading the cached applet is a problem in two situations:

If you load a new version of the ESS microcode during that time (while you have not yet
quit your browser), the cached applet might not conform to the new code on the ESS.

If unauthorized users have access to your workstation when you leave it, they can
access the ESS through your cached user ID.
Note: This is available only if the FlashCopy feature V2 is installed. If the FlashCopy V1
is installed, then the source and target volumes
must
be in the same LSS.
Note: This is available only if FlashCopy V2 is installed. If FlashCopy V1 is installed,
then the source can only be in one FlashCopy relationship at a time.
Note: Prior to establishing a FlashCopy, we recommend that you do the following:
1.Unmount the target volume from all host systems, since the FlashCopy process is a
destructive operation to the target and will overwrite the data on the target volume.
2.Additionally, stop all active reading/writing operations to the source.
3.Also, quiesce your application and flush the data to the source volume.

226
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.11.1 Establishing a FlashCopy pair
Use the Volumes panel to establish a FlashCopy pair. In the source area, select the LSS
within which you will select the source volume. To select the LSS, you will use the source
drop-down list. When the source LSS is selected, select the LSS within which you will select
the target volume in the target area.
You always need to have two components to establish a FlashCopy pair: a source and a
target. Click on the source volume to select it, then right-click on the target volume to select
the volume. If you have selected the wrong source or target volume, just click on the correct
source volume to start again.
The source and target logical subsystem is specified the following way: Device type (4
digits):ESS Serial number (5 digits):LSS number (2 digits).
In our example shown in Figure 6-34, we have selected 2105:22331:10 as the source LSS
and 2105:22331:11 as the target LSS. The volume 000 in LSS 10 is selected as a source
volume and volume 000 in LSS 11 is selected as target volume.
Figure 6-34 Volumes panel with source and target volumes selected
Another right-click on the target volume will bring up the Task Wizard panel. In the Select task
type panel, select the Establish FlashCopy option (see Figure 6-35 on page 227). Click on
Next.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
227
Figure 6-35 Task Wizard panel
Select the options for the FlashCopy task, as shown in Figure 6-36, and described in the
following paragraphs.
Figure 6-36 Define task panel
Selecting the FlashCopy options
Refer to 3.4, “FlashCopy tasks and options” on page 41 for a detailed description of the
different options for FlashCopy and to fully understand the different options. We present only
a summary here of the different options.
For the following descriptions, refer to Figure 6-36.
No background copy option (nocopy)
If this option is checked:

Only the tracks modified on the source volume are copied to the target volume.

228
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

This relationship between source and target volume remains forever and has to be broken
manually.

By default, this option is not selected and all data is copied from the source to the target
volume of a FlashCopy pair (background copy). Once all data is copied, the relationship
ends automatically (unless the Persistent FlashCopy option is selected).

If two volumes are already in a FlashCopy
nocopy
relationship, you can perform a
FlashCopy Start Background Copy to create a physical background copy.
Accelerated destage mode
Select this option to cause a FlashCopy source volume track, which is being modified and is
in a FlashCopy relationship, to be destaged from cache sooner than it would be if normal
cache algorithms were applied. This feature minimizes (for the source volume) the number of
modified tracks that are resident in cache.
Note that:

With FlashCopy Version 1, this option might (under specific circumstances) speed up the
completion of the background copy process.

With FlashCopy Version 2, this feature has no effect, since the cache algorithms have
changed.
Permit establish if target is online
Select this option if you want to establish a FlashCopy relationship, even if the target is online
to the S/390 or zSeries host.
This option does not apply to open-systems volumes.
Persistent FlashCopy
Select this option if you want the FlashCopy relationship to remain even after the FlashCopy
operation completes. This prevents another FlashCopy task from writing on your target
volume before you have withdrawn the FlashCopy relationship. The FlashCopy relationship
between source and target volumes remains indefinitely and must be broken with a Withdraw
FlashCopy task.
Inband command
Select this option to allow a FlashCopy relationship between two volumes to be established at
a remote ESS using inband commands (the command will be sent over an established PPRC
path). For details, refer to 3.4.3, “FlashCopy V2 establishment options” on page 43 and
6.11.11, “Inband FlashCopy” on page 256.
Freeze FlashCopy Consistency Groups
Select this option to freeze write activity to each LSS containing FlashCopy source volumes
that are in a Consistency Group. A Consistency Group is a group of volumes participating in
FlashCopy relationships that need to be kept in a consistent state to ensure data integrity.
The freeze option provides a mechanism to ensure data consistency across multiple
FlashCopy volumes in an LSS, or across LSSs, or across Model 2105 ESSs. The freeze
option causes volumes to remain in a
Queue Full
condition (volumes are not available) until
the condition is reset or the time-out value expires (default is two minutes).
Inhibit writes to target
Select this option to ensure that host writes are inhibited on the target volume until an
Incremental FlashCopy operation is complete. This ensures data consistency on the target

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
229
volume. If you select the Inhibit writes to target option, the change recording option will not be
effective on the target volume.
Start change recording
Select this option to create the bitmaps that record the changed tracks on both volumes
participating in a FlashCopy relationship. For details, refer to 3.4.3, “FlashCopy V2
establishment options” on page 43 and 6.11.6, “Incremental FlashCopy” on page 240.
Increment FlashCopy
Select this option to create a new point-in-time data copy without copying all of the tracks from
the source volume to the target volume. This feature provides an alternative to copying an
entire volume for each point-in-time data copy. For details, refer to 3.4.3, “FlashCopy V2
establishment options” on page 43 and 6.11.6, “Incremental FlashCopy” on page 240.
Reverse Restore
Select this option to reverse the FlashCopy relationship and copy the modified tracks from the
target volume to the source volume. The background copy process must complete before you
can reverse the order of the FlashCopy relationship from its original source and target
relationship. For details, refer to 3.4, “FlashCopy tasks and options” on page 41 and 6.11.6,
“Incremental FlashCopy” on page 240.
Fast Reverse Restore
This option can only be used if the FlashCopy was established using Start Change
Recording, Persistent FlashCopy, and only one FlashCopy from the source. This method
does not require a Background Copy; only the changed data is copied back to the source.
The target is left in an indeterminate state and data must be rewritten to it. This process is
primarily used to recover from an Asynchronous PPRC disaster event. Asynchronous PPRC
is discussed in 4.5, “Asynchronous PPRC” on page 74.
Revertible
This option is included on this panel for compliance with Asynchronous PPRC only. It is not a
stand-alone option and should not be selected by the user. If you wish to use this option, call
your IBM representative for technical support.
Running or saving the task
Once you select the FlashCopy options, click on Next. From the following panel, as in
Figure 6-37 on page 230, you will decide whether to Save, Run, or Cancel the task.

230
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-37 Define task panel
If you save the task, then it can be executed again from the task panel at any time.
When the FlashCopy is issued, a bitmap is created for the data copy from the source to the
target volumes. The time to build this bitmap is only a few seconds (depending on the size of
the volumes). Once the bitmap is created, the FlashCopy relationship is established and the
two volumes are immediately available.
Once the FlashCopy relationship is established, the display of the volumes in the volume
panel changes (see Figure 6-38):

A blue lightning bolt appears next to the FlashCopy source volume.

A red lightning bolt appears next to the FlashCopy target volume.

A
I-am-a-target
icon appears to the right of the red lightning bolt next to the target volume.
This icon is a small gray square. The icon remains when the FlashCopy relationship ends,
for example, after a background copy is finished. To remove this icon, you must withdraw
the FlashCopy relationship with a
withdraw to target
task (see 3.4.4, “FlashCopy V2
withdraw options” on page 44).

The lightning bolts disappear when the copy operation is complete. If the FlashCopy
relationship is established with a No copy option or with the Persistent FlashCopy
option, the icons remain until a withdraw operation is performed.

You can get information about the progress of the FlashCopy background copy by clicking
on the Information Panel button.
Figure 6-38 FlashCopy lightning bolts icons

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
231
Establishing FlashCopy relationship using the Logical Subsystem panel
To establish FlashCopy relationships between all the volumes in two LSSs, you can create a
task at the LSS level in the Logical Subsystem panel (LSS panel).
Open the Logical Subsystems panel and select the ESS in the Select list in the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystems panel
do have different meanings (for further information, refer to Table 6-2 on page 209).
Click on the source LSS and the LSS ID turns blue. Right-click on the target LSS and the LSS
ID turns red. Right-click again on the target LSS and the Task Wizard opens. Select the
options as you would do for volume pairs.
FlashCopy icons
The icons show the status of the FlashCopy pair and will be displayed next to the volume icon
on the volumes status view. A list of these icons with a brief description can be displayed at
any time by clicking the Legend button on the volume status screen. See Table 6-1 on
page 201 for the icons.
6.11.2 Getting information about a FlashCopy pair
By selecting one volume in the Volumes panel and clicking the Information Panel button, you
get information about the state of the FlashCopy pairs the volume is involved in.
In the example in Figure 6-39 on page 232, we selected a volume that is the source for two
FlashCopy relationships. The target volumes IDs are in the Full Volume FlashCopy Peers
panel.
Tip: After you have created and saved the task to establish the FlashCopy with the
No background copy option, you need to create the corresponding task to withdraw the
pair. We recommend that you modify the task that you just created so that it withdraws the
pair, then name and save the task. This not only saves time, but it is also useful for
speeding up Disaster Recovery. For further information, refer to 6.8.5, “Modifying a task”
on page 221.

232
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-39 Volume Information Panel
6.11.3 Withdrawing a FlashCopy pair
In the following cases, you have to
withdraw
a FlashCopy pair:

If a FlashCopy pair is not needed anymore, but has not yet finished the background copy.

If a FlashCopy pair that was created with the Do not perform background copy option is
not needed anymore.

If a FlashCopy pair that was created with the Persistent FlashCopy option is not needed
anymore.
The option to Withdraw FlashCopy pair results in the termination of the FlashCopy
operation. The target volume is now available to the host systems as normal, which then may
alter data.
You can use the Volumes panel or the Logical Subsystem panel to withdraw a FlashCopy pair.
Open the Volumes panel and select the source and target LSS within which you want to
perform the withdraw FlashCopy in the Source and Target pull-downs. You always need to
have selected two components to withdraw a FlashCopy pair, a source and a target. Select
the source volume and right-click on the target. If you have selected the wrong source or
target volume, just click on the correct source volume again. See Figure 6-40 on page 233 for
an example.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
233
Figure 6-40 Volumes panel: Select volumes
Once the source and target are selected, start the Task Wizard (Figure 6-41) by right-clicking
on the target volume again.
Figure 6-41 Task Wizard: Select task type
Select the Withdraw FlashCopy pair option and click on Next to select if you want to issue
this command using an established PPRC path (inband command). See 6.11.11, “Inband
FlashCopy” on page 256 for more details.

234
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-42 Task Wizard: Select withdraw options
Click on Next and decide whether to Save, Run, or Cancel the task (see Figure 6-43).
Figure 6-43 Task Wizard: Define Task
Withdrawing FlashCopy pairs using the Logical Subsystem panel
To withdraw all FlashCopy pairs between two logical subsystems, you can create a task at the
LSS level in the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystems panel
do have different meanings (for further information, refer to Table 6-2 on page 209).
Click on the source LSS and the LSS ID turns blue. Right-click on the target LSS and the LSS
ID turns red. Right-click again on the target LSS and the Task Wizard opens. Select the
options as you would do for volume pairs.
Warning: All the FlashCopy relationships that have source volumes in the selected source
LSS and their associated targets in the selected target LSS will be terminated.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
235
Withdraw FlashCopy to the target
For this option, only the target volume is selected. Use the option Withdraw FlashCopy to
the target against the target volume when you want to remove its FlashCopy icon
(represented by a white square at the lower left of the volume icon; see Table 6-1 on
page 201) after the background copy has been successfully completed. Typically, you would
remove the indicator because you have decided that it is no longer necessary to mark the
volume as a copy of another.
To perform this action, click on the target volume and right-click on the target volume. The
volume icon and the volume ID turns gray (see Figure 6-44).
Figure 6-44 Volumes panel: Selecting a volume as source and target for FlashCopy withdraw
Right-click on the target volume again to open the Task Wizard (Figure 6-45 on page 236).
Attention: If you perform a Withdraw FlashCopy to target action to a target volume of an
established relationship, it will withdraw the relationship and remove the
I-am_a_target

icon.

236
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-45 Task Wizard: Select task type
Select the Withdraw FlashCopy pair option and then click Next to select the withdraw
options (Figure 6-46).
Figure 6-46 Task Wizard: Select withdraw options
Click on Next and decide whether to Save, Run, or Cancel the task (Figure 6-47).
Figure 6-47 Task Wizard: Define Task

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
237
Withdrawing FlashCopy to the target using the Logical Subsystem panel
To withdraw all FlashCopy relationships to the target for all target volumes in the same logical
subsystem, you can create a task at the LSS level in the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystems panel
do have different meanings (for further information, refer to Table 6-2 on page 209).
Click on the LSS and the LSS ID turns blue. Right-click on the same LSS and the LSS ID
turns gray. Right-click again on the LSS and the Task Wizard opens. Select the options as you
would do for volumes pairs.
6.11.4 Multiple FlashCopy using a single source volume
Multiple FlashCopy using the same volume is possible with LIC 2.2.0 or higher and the
FlashCopy V2 feature installed. FlashCopy V2 enables you to have multiple FlashCopy
relationships using the same source volume.
To do this, create different FlashCopy tasks using the same source volume, and then
establish each FlashCopy relationship by running each individual task.
When a volume is the source of more than one established FlashCopy relationship, a specific
Multiple Relationship FlashCopy Icon will appear next to the FlashCopy Source icon, as
shown in the Figure 6-48.
Figure 6-48 FlashCopy source and Multiple Relationship FlashCopy icons
6.11.5 FlashCopy Start Background Copy
If you have performed a FlashCopy pair with the No background copy option and you need
to create a permanent physical copy of the data for backup or disaster recovery purposes,
you can perform a FlashCopy Start Background Copy. All data will be copied from the
Note: If you perform the Withdraw FlashCopy to the target, make sure you select a
target volume of a FlashCopy relationship in the volumes panel. You cannot perform this
action by selecting a source volume.
Note: All FlashCopy relationships that have a target volume in the LSS that you select will
be terminated.
Restriction: The maximum FlashCopy relationships a volume can be the source for is 12.
Restriction: Among the Multiple FlashCopy relationships the volume is the source for, only
one can be a FlashCopy relationship with the Start Change Recording option (this option
is a prerequisite prior to establishing an Incremental or a Reverse Restore FlashCopy
relationship).

238
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
source to the target. After the copy is complete, the FlashCopy relationship is terminated,
unless the FlashCopy relationship is Persistent, in which case, the relationship remains.
FlashCopy Start Background Copy using the Volumes panel
Open the Volumes panel and select the source LSS and the target LSS in the source and
target panels.
To perform a FlashCopy Start Background Copy, click on the source volume of the
FlashCopy pair. Then right-click on the source volume (the target ID of the volume turns gray,
as presented in Figure 6-49).
Figure 6-49 Volumes panel: The target volume selected as source and target
Then, right-click on the target volume again to open the Task Wizard (Figure 6-50 on
page 239).
Note: If you perform FlashCopy Start Background Copy, make sure you select a source
volume of a FlashCopy relationship in the volumes panel. You cannot perform this action
by selecting a target volume.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
239
Figure 6-50 Task Wizard: Select task type
Select the Withdraw FlashCopy pair option and click Next. In the next panel, you select the
FlashCopy Start Background Copy option (Figure 6-51).
Figure 6-51 Task Wizard: Select withdraw options
Click the Next button and decide whether to Save, Run, or Cancel the task (Figure 6-52 on
page 240).

240
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-52 Task Wizard: Define Task
Once you run the task, the background copy task copies the data from the source to the
target. After the copy is complete, the FlashCopy relationship is terminated and the lightning
bolts disappear on the source and the target volumes, unless the FlashCopy relationship is
persistent, in which case, the relationship remains.
FlashCopy Start Background Copy using the LSS panel
To start a background copy of all FlashCopy pairs that have a source volume on a logical
subsystem, you can use the Logical Subsystem panel.
Open the Logical Subsystems panel and select the ESS in the Select list at the upper right
corner. A panel showing all the LSSs is displayed, with the appropriate name and serial
number indicated below each LSS icon. The different icons on the Logical Subsystem Panel
do have different meanings (for further information, refer to Table 6-2 on page 209).
Click on the desired LSS. The LSS ID turns blue. Right-click on the LSS and the LSS ID turns
gray. Right-click on the LSS again and the Task Wizard opens. Select the options as you
would do for volumes pairs.
6.11.6 Incremental FlashCopy
Incremental FlashCopy is possible with LIC 2.2.0 or higher and the FlashCopy V2 feature
installed. With the Incremental FlashCopy function, it is no longer necessary to copy an entire
volume for each point-in-time copy. Instead, only tracks that have changed on the source
volume since the last Incremental FlashCopy are copied to the target volume. To enable
Incremental FlashCopy, the ESS uses a feature called
change recording
to monitor writes and
record changes that were made to volumes participating in the FlashCopy relationship since
the initial (or last) Incremental FlashCopy was performed.
A source volume can participate in multiple (up to 12) FlashCopy relationships. However,
when you establish a FlashCopy pair with the Start Change Recording option on a volume
Note: When you run the task at the LSS level, be aware that the task will run on all the
source volumes in that LSS.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
241
pair, the option only applies to that specific pair, regardless if the same source volume is
participating in multiple FlashCopy relationships. Incremental FlashCopy operations (with the
Start Change Recording featured enabled) are performed on a one-to-one basis, that is,
incremental changes are copied from one source volume to one target volume.
The following describes how you can enable an Incremental FlashCopy:

The initial setup requires that you enable the Start Change Recording option when you
establish the initial FlashCopy volume pair. This process copies the entire source volume
to the target volume to keep the volumes in sync and activates the change recording
feature. See “Initial task: Establishing a FlashCopy pair to start change recording” on
page 241 for instructions.

Each time you want only incremental updates made to the target volume, you must
establish the FlashCopy operation with the Increment FlashCopy option. Note that the
Start Change Recording option should also be selected. The ESS copies only data that
has changed since the last incremental copy and avoids recopying data that has not
changed. See “Establishing an Incremental FlashCopy” on page 244 for instructions.
Initial task: Establishing a FlashCopy pair to start change recording
Open the Volumes panel and select the source LSS in the left panel, then select the target
LSS in the right panel. Select the source volume and, with a right-click, select the target
volume. With another right-click on the target volume, you will open the Task Wizard. Select
the Establish FlashCopy pair option (Figure 6-53 on page 242), and then click on Next.
Note:

If you perform an Incremental FlashCopy on a FlashCopy pair without the required
change recording activated, the operation fails. You must withdraw the FlashCopy
relationship, then reestablish the FlashCopy relationship with the Start Change
Recording enabled.

The Persistent FlashCopy option is enabled for Incremental FlashCopy operations.
Because the FlashCopy relationship between source and target volumes remains
indefinitely, you can withdraw the relationship with a Withdraw FlashCopy pair task.

242
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-53 Volumes panel: Select volumes
In the next panel, you have to select the Start Change Recording option (Figure 6-54). This
option will enable you to run an

Incremental FlashCopy task or a Reverse Restore FlashCopy
task later using these two volumes. When you select the Start Change Recording option,
the Persistent FlashCopy option is automatically selected (Figure 6-54). You can also select
other options if needed.
Figure 6-54 Task Wizard: Select copy options

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
243
Once the options are selected, click on Next and decide whether to Save, Run, or Cancel the
task (Figure 6-55).
Figure 6-55 Task panel: Define Task
Once the FlashCopy is established, a new icon will appear next to the FlashCopy lightning
bolts in the source and target volume. This new icon is a yellow and red lightning bolt above a
green triangle (see Figure 6-56 for an example). This Icon means that the Start Change
Recording option is enabled on the source and target volumes and that the volumes are
eligible for an Incremental FlashCopy.
Figure 6-56 Volumes panel: Result of a Start Recording FlashCopy

244
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Establishing an Incremental FlashCopy
You can perform an Incremental FlashCopy using two volumes in a FlashCopy relationship
with Start Change Recording enabled. To verify that the Start Change Recording is enabled
on a FlashCopy pair, you can select the source volume and click on Information Panel to
check for the FlashCopy properties (Figure 6-57).
Figure 6-57 Information panel: Source volume and start recording
Open the Volumes panel and select the source LSS in the left panel, then select the target
LSS in the right panel. Select the source volume and, with a right-click, select the target
volume. With another right-click on the target volume, you will open the Task Wizard. Select
Establish FlashCopy pair option (Figure 6-58 on page 245), and then click on Next.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
245
Figure 6-58 Volumes panel: Selecting volume for an Incremental FlashCopy
Select the Incremental FlashCopy and Start Change Recording options. The Persistent
FlashCopy option is automatically selected when you click on the Start Change Recording
option (Figure 6-59 on page 246).
Important: When you perform an Incremental FlashCopy operation, change recording is
not re-enabled by default. You must select the Start Change Recording option each time
you establish an Incremental FlashCopy operation to ensure that change recording is
maintained on the volume pair. If you do not select the Start Change Recording option,
you will not be able to run another Incremental FlashCopy or a Reverse Restore
FlashCopy after you run the task.

246
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-59 Task Wizard: Select copy options
Click on Next and decide whether to Save, Run, or Cancel the task (Figure 6-60).
Figure 6-60 Task Wizard: Define Task
6.11.7 Reversing a FlashCopy relationship
Reversing a FlashCopy is possible with LIC 2.2.0 or higher and the FlashCopy V2 feature
installed. When used within an established FlashCopy pair, this option allows you to restore
the source volume with the data from the target volume. To get more details about how the
reverse restore functions operates, refer to 3.3.3, “Incremental FlashCopy and the Reverse
Restore option” on page 38.
You can perform a Reverse Restore FlashCopy using two volumes in a FlashCopy
relationship with Start Change Recording enabled and only if the background copy between
the two volumes is finished.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
247
Also available from LIC 2.4.0 onward is Fast Reverse Restore. Its use is also discussed in
this section.
To verify that the Start Change Recording is enabled on a FlashCopy pair and that the
background copy is finished, you can select the source volume and click on the Information
Panel to check for the FlashCopy properties. The number of Sectors out of sync must be 0,
indicating that the background copy is finished (Figure 6-61).
Figure 6-61 Change Recording enabled: Background copy finished
Open the Volumes panel and select the original source LSS in the left panel, then select the
original target LSS in the right panel. Select the source volume and, with a right-click, select
the target volume. With another right-click on the target volume, you will open the Task
Wizard. Select the Establish FlashCopy pair option (Figure 6-62 on page 248).
Important: The FlashCopy is reversing direction, but you do not need to swap the pairs in
your view window. The original source is opened on the left and the original target on the
right. The
reverse restore
command will do the reverse for you.

248
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-62 Volumes panel: Selecting volume for a Reverse Restore FlashCopy
Once the Establish FlashCopy pair task is selected, click on Next.
Select the Reverse Restore and Start Change Recording options. The Persistent
FlashCopy option is automatically selected when you click on Start Change Recording
(Figure 6-63).
Figure 6-63 Task Wizard: Select copy options
Important: If you do not select the Start Change Recording option, you will not be able to
run another incremental or another Reverse Restore FlashCopy in the future after you run
this task.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
249
Click on Next and decide whether to Save, Run, or Cancel the task (Figure 6-64 on
page 249).
Figure 6-64 Task Wizard: Define Task
Fast Reverse Restore
Fast Reverse Restore should be used primarily for Asynchronous PPRC fatal recovery. Use
this option to restore the FlashCopy source from its FlashCopy target without having to wait
for the background copy from the original source to the original target to complete.
Prerequisites
The following must be true to be able to perform a Fast Reverse Restore operation. The
FlashCopy pair must have been established with the following options:

Start Change Recording


No Background Copy


Persistent FlashCopy


Inhibit Write to Target
The procedure is the same as the Reverse Restore above, but select the Fast Reverse
Restore option on the panel, as shown in Figure 6-65 on page 250, instead of Reverse
Restore.

Do not select No Background Copy (so that the background copy is performed).

Do not select Change Recording.

Do not select Persistent.

Select Fast Reverse Restore (note that the Reverse Restore option will also be checked
for you).
There should be no I/O to either volume during the fast reverse restore.
Important: The original target contents are unpredictable after the Fast Reverse Restore
is complete.

250
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-65 Fast Reverse Restore options
The status of the FlashCopy will be as shown in Figure 6-66 on page 251, while the reverse
copy is in progress, and when complete all status icons will be removed to indicate no
FlashCopy relationship exists. Note that although the old target is available for use, the data is
unpredictable and the entire volume must be rewritten.
Tip: At this time, it is a good idea to perform a FlashCopy back in the original direction.
This restores the data on the target to a known good copy and also gives you another copy
of the data in case the process has to be repeated or the data on the original target is
corrupted during failed recovery attempts.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
251
Figure 6-66 Fast Reverse Restore in progress
6.11.8 Creating a FlashCopy Consistency Group
The FlashCopy V2 feature also has two new options: the Freeze FlashCopy pairs and
FlashCopy Consistency created options. You can combine those two options to create a
Consistency Group across several volumes in different LSSs and even in different ESSs in
the same Copy Services Domain. Refer to 3.4.5, “Consistency Created task with FlashCopy
V2” on page 45 to get more details about how this options works.
Here are the steps required to build a FlashCopy Consistency Group:
1.Create a task to Freeze a FlashCopy Consistency Group. To do this, you will:
a.Create and save tasks with the Freeze options for each FlashCopy pair.
b.Group all the tasks into one single Freeze task.
2.Create a task to Thaw the FlashCopy Consistency Group. To do this, you will:
a.Create a task with FlashCopy Consistency Group created option for each LSS
containing source volumes.
b.Group all the tasks into one single Consistency created task.
3.Invoke the Freeze task to freeze the FlashCopy Consistency Group. You can invoke the
task using the Copy Services Web User Interface or the CLI.
4.Invoke the Consistency created task. You can invoke the task using the Copy Services
Web User Interface or the CLI.
6.11.9 Freezing a FlashCopy Consistency Group
The FlashCopy Establish command can freeze write activity to FlashCopy source volumes in
a Consistency Group. The freeze option ensures data consistency across multiple FlashCopy
volumes in an LSS, across LSSs, or across Model 2105 ESSs.

252
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Considerations

You create a FlashCopy Consistency Group by grouping several FlashCopy tasks with the
Freeze FlashCopy Consistency Group option specified within one single group task and
running this group task. If you want to create different FlashCopy Consistency Groups, you
will create different group tasks.

When you establish a FlashCopy operation (with the Freeze FlashCopy Consistency
Group option specified), the freeze is issued to each volume in the Consistency Group.
This causes the volumes to remain in a
Queue Full
condition (volumes are not available to
the application) until the condition is reset (see 6.11.10, “Thawing a FlashCopy
Consistency Group” on page 255) or the time-out value expires (The default is two
minutes). You can use the LSS properties panel to modify the default value (see 6.6.4,
“LSS properties” on page 212).

When source volumes are in a frozen state, and after a FlashCopy operation (with the
freeze option specified) has been successfully issued to each volume in the Consistency
Group, the FlashCopy target volumes will be in a consistent state.
To create tasks that will freeze a FlashCopy Consistency Group:
1.Use the volume panel to build the tasks that will establish the FlashCopy relationship with
a Freeze FlashCopy Consistency Group option. Build one task for each FlashCopy pair.
You can use the Multiple selection mode when selecting several volumes in the same
LSS.
2.Use the task panel to group the FlashCopy tasks with the freeze option into one single
task for all options that need to be in the Consistency Groups.
Establishing a FlashCopy with freeze option
In the volume panel, in the source area, select the LSS that contains the source volume. To
select the LSS, you will use the source drop-down list. When the source LSS is selected,
select the LSS that contains the target volume in the target area.
Click on the source volume. Once the source volume is selected, right-click on the target
volume. You can use the Multiple Selection Mode to select multiple source and target volume
pairs (Figure 6-67 on page 253).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
253
Figure 6-67 Volumes panel: Selecting volumes using Multiple Selection Mode
To open the Task Wizard, right-click again on the last target volume you selected.
In the Select Task type panel, select the Establish FlashCopy pair task type (Figure 6-68).
Figure 6-68 Task Wizard: Select task type
In the Select copy options panel, click on Freeze FlashCopy Consistency Group.
Eventually, select other copy options for the task (Figure 6-69 on page 254), and then click on
Next.

254
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-69 Task Wizard: Select copy options panel
In the Define task panel (Figure 6-70), either give a task name (mandatory) and a task
description (optional), then click on Save to run the task later, or click on Run to run the task
immediately.
Figure 6-70 Task Wizard: Define Task panel
Grouping tasks
1.In the task panel, select all the FlashCopy tasks you want to group. Use the Shift and Ctrl
keys to select multiple tasks.
2.Click on a group.
3.Give the task a name (mandatory) and a description (optional), and then save the task
group.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
255
6.11.10 Thawing a FlashCopy Consistency Group
The thawing of a FlashCopy Consistency Group (Consistency Group created operation)
follows the successful freeze for all FlashCopy volumes in the Consistency Group. It allows
normal I/O operations from the host to the ESS to resume by removing the
Queue Full

condition on a group of FlashCopy source volumes.
Considerations
The Consistency created task is performed at the LSS level. This task will resume the
Queue
Full
condition for all the volumes in the LSS. When the source volumes in the Consistency
Group are spread across LSSs or across ESSs, you will need to perform the following
actions:

Create a Consistency Created task on each LSS containing a FlashCopy source with the
Queue-full condition.

Group all the tasks into one single task.

Run the task.
Creating a Consistency Created task
In the LSS panel, select the ESS using the Select drop-down menu. Click on one LSS and
right-click on the same LSS. The LSS number will turn gray. Right-click again on the LSS.
In the Select task type, select Consistency Created (Figure 6-71), and then click Next.
Figure 6-71 Task Wizard: Select task type
In the Select copy options panel, select FlashCopy Consistency Group (One LSS
selected) (Figure 6-72), and then click Next.
Figure 6-72 Task Wizard: Select copy options

256
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
In the Define task panel (Figure 6-73), either give a task name (mandatory) and a task
description (optional) and then click on Save to run the task later, or click on Run to run the
task immediately.
Figure 6-73 Task Window: Define Task
Grouping tasks
1.In the task panel, select all the FlashCopy tasks you want to group. Use the Shift and Ctrl
keys to select multiple tasks.
2.Click on a group.
3.Give the task a name (mandatory) and a description (optional), and then save the task
group.
6.11.11 Inband FlashCopy
You can establish a FlashCopy pair at a remote ESS using inband commands. Inband
commands allow you to establish and withdraw a FlashCopy pair at the remote site, without
having a Copy Services Web interface connection to the remote site (this connection requires
a LAN connection).
Inband commands are issued to a primary volume of a PPRC pair at a local ESS and sent
across PPRC paths (acting as a conduit) to a remote ESS to enable a FlashCopy pair to be
established at the remote site. The source volume you select for the FlashCopy at the remote
site must be the secondary PPRC volume of the PPRC pair. (The PPRC secondary volume
becomes the source volume in a FlashCopy pair.)
The task definition process is the same as described in 6.11.1, “Establishing a FlashCopy
pair” on page 226. To create a task that will use an inband command to invoke the task at the
remote site, you must select the Inband Command option.
Important: When you create the FlashCopy tasks with the Inband option, the remote ESS
must be a member of the same ESS Web domain as the PPRC pair. The connection is not
mandatory when you issue the command (when you run the task) but is mandatory when
you define (create) the command.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
257
Here are the steps for creating an Inband FlashCopy:
1.In the Volumes panel, select the source volume (with a left-click) and target volume (with a
right-click) for your FlashCopy relationship. Then right-click the target volume again to
open the Task Wizard and select the establish FlashCopy pair task (Figure 6-74).
2.When you run the task, the source volume must be the secondary volume of a PPRC or
PPRC-XD pair (Figure 6-74).
Figure 6-74 Volumes panel: Volume selection for a FlashCopy with Inband command option
3.Select the Establish FlashCopy pair option and click Next. Within the next panel, you
select the Inband Command option and the other options for your FlashCopy relationship
(in our example,Figure 6-75, we also select the No Background Copy option).
Figure 6-75 Task Wizard: Select copy options

258
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
4.Click the Next button and choose from which LSS you will issue the inband command
(Figure 6-76). The LSS you select is on the local site. It has to be the LSS where the
primary volume of the PPRC relationship resides.
Figure 6-76 Task Wizard: Choose LSS
5.Click the Next button and decide whether to Save, Run, or Cancel the task (Figure 6-77).
Figure 6-77 Task Wizard: Define Task
6.Once you run the task, the active server for Copy Services will issue the command to the
LSS selected when creating the task. The order to run the task will be sent to the remote
site from this LSS using an active path to the remote ESS. After the FlashCopy is
established, the lightning bolts will appear on the source and the target volumes
(Figure 6-78 on page 259).
Important: You must have a PPRC relationship running between the primary and
secondary site. The FlashCopy with the Inband command will only run successfully if the
source volume for the FlashCopy is the secondary (target) volume of an established PPRC
session (it can be suspended, as long as the links are available). The FlashCopy Inband
command will not run successfully if the source volume is in simplex state.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
259
Figure 6-78 Volumes panel: Result of a FlashCopy with Inband Command option
6.12 PPRC setup examples using the Web User Interface
In the following sections, we explain how to set up and configure different PPRC
environments using the ESS Copy Services Web User Interface. The Web User Interface
panels illustrated in these sections are the panels that a user will see if at a minimum level of
LIC 2.3.0.
In general, there are two steps needed to successfully establish PPRC relationships:

Setting up paths between the PPRC source and target LSSs

Establishing the PPRC pairs, either on single volumes or on entire logical subsystems
6.13 Paths
Be aware of these requirements of the PPRC functionality:

Paths for PPRC have to be available and need to be defined and established first.

There are some rules when establishing paths (see 4.9, “PPRC connectivity” on
page 101).

The target volume has to be the same size as the source or larger.
Important note: To set up all PPRC pair related functions, either the Volumes panel or the
Logical Subsystems panel (all volumes in the source and the target LSS) can be used. In
this book, only the usage of the Volumes panel is explained. For further information about
how to set up a PPRC pair related function on the Logical Subsystem panel, refer to the
IBM TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.

260
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.13.1 Establishing paths
Before you can establish any PPRC pairs, you first have to set up the paths between the
source and the target LSS where the primary and secondary volume reside. The paths are
needed for communication between the PPRC pairs and to copy data from the source to the
target.
Use the Paths panel of the ESS Copy Services Web User Interface to set up paths for PPRC
and select Work with Paths, as shown in Figure 6-79. Then follow this sequence:
1.Select the source ESS. The LSSs are then displayed.
2.Select an LSS. Once the LSS is selected, you can then select the target ESS.
3.From the drop-down menu of the Logical Subsystem Targets column, select the target
ESS.
4.Select the target LSS. Notice that you can now select ESCON or FCP.
5.Select FCP or ESCON from the Local Port/Remote Port column.
6.Click on the first FCP or ESCON adapter and right-click on the subsequent adapters.
Figure 6-79 Establishing paths
The ESSs that are connected are automatically displayed in the Logical Subsystem Targets
area.
Note: Multiple adapters could be selected with a right-click after the first adapter was
selected. If you choose the wrong adapter(s), just click on the correct adapter again to
delete the selection.
2
1
3
4
5
6

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
261
In the example shown in Figure 6-79 on page 260, we will establish an FCP path between the
source LSS 17 in ESS 23953 and the target LSS 16 in ESS 24663.
Once the target and source of the PPRC path have been selected, right-click on one of the
highlighted adapters to bring up the Task Wizard (Figure 6-80). Select the Establish paths
option and click Next.
Figure 6-80 Task Wizard: Select task type
If a switch is part of the path, the Select outgoing ports panel of the Task Wizard opens, as
shown in Figure 6-81, when using ESCON links.
Figure 6-81 Select outgoing port: ESCON link
Note: If you have some ESSs that are directly connected (that is, no switch between
them), you can click Display Direct Connect Paths to display those paths.
You must perform a refresh function (see 6.9.2, “Refreshing path and volume information
on an ESS” on page 224) for the following conditions:

If this is the first time you are accessing ESS Copy Services

After an update of the ESS microcode

After a concurrent service action

If the cabling for the PPRC links is modified

262
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
If you are working with Fibre Channel links and a switch is present, then the Select outgoing
ports Task Wizard will be similar to Figure 6-82.
Figure 6-82 Select outgoing port: FCP link
In the outgoing ports list, select the desired port on the switch. Click Next. In the next panel
you specify the path options, as Figure 6-83 illustrates, when working with ESCON links:

Do not establish paths if they already exist: If this option is checked and there is
already a path defined from the source to the target, the operation of establishing paths
will not be executed. Does not apply to FCP paths.

Force removal of existing paths: You must select this option if the selected path goes
through a switch. The selection causes any logical paths between a host system and the
selected ESCON adapter to be removed before establishing logical paths. Does not apply
to FCP paths.
Figure 6-83 Select path options: ESCON links
If working with FCP links, the paths options you will see are shown in Figure 6-84 on
page 263.
Attention: The Force removal of existing paths option removes all logical paths from
the host to the port. This is potentially disruptive to host activity, since there may be
applications using the port. By host command under operator control, you should vary
offline all logical paths from the host to the port. Only as a last solution or when the
operator is absolutely sure there would be no negative impact should the Force
removal of existing paths option be used.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
263
Figure 6-84 Select path options: FCP links
In either case, ESCON or FCP, you can also select PPRC Consistency Group to create a
Consistency Group. For information on Consistency Groups, see 4.5, “Asynchronous PPRC”
on page 74.
Click Next and decide whether to Save, Run, or Cancel the task. Click Save if you want to
save the task in the task repository. If you save the task, it can be executed from the Tasks
panel at any time.
Figure 6-85 Task Wizard: Define Task
When you run the task, and if the PPRC path is successfully established then three blue
asterisks appear directly below the adapter in the Local Port/Remote Port column of the
Paths panel, as illustrated in Figure 6-86, when using an ESCON link.
Tips:

If you save the task, it can be executed from the Tasks panel at any time (see 5.7.3,
“Running a task and viewing an error about a failed task” on page 146).

After you create and save the task to establish paths, we strongly suggest that you
immediately create the corresponding task to remove the path. You can modify the task
within the Tasks panel so that it removes the path (see 6.8.5, “Modifying a task” on
page 221). This way, you will save time when creating your tasks for Disaster Recovery.

264
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-86 ESCON path successfully established
Figure 6-87 illustrates the display when a path is successfully established using an FCP link.
Figure 6-87 FCP path successfully established
6.13.2 Removing paths
You can remove paths using the Paths panel. The blue asterisks below the path-connection
symbol in the Local Ports/Remote Ports column mean that a path is already established. This
is how you remove a path (refer to Figure 6-88 on page 265):
1.Select Work with Paths.
2.From the drop-down menu, select the ESS source.
3.Click on the desired LSS source from the Logical Subsystem Sources panel.
4.From the drop-down menu, select the ESS target.
5.Click on the desired LSS target from the Logical Subsystems Target panel.
6.From the drop-down menu in the adapter column, select either ESCON or FCP.
7.Click on the path to be removed (it should have ***).
8.Right-click on the path to remove.
Note: You can see information about the established path using the Information Panel.
Refer to 6.7.5, “Path Information Panel button” on page 217.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
265
Figure 6-88 Selecting an established path to be removed
The Task Wizard opens (see Figure 6-89).
Figure 6-89 Task Wizard: Select task type
Select the Remove paths option and click Next. In the next panel, you can specify the task
options (see Figure 6-90 on page 266).

266
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-90 Task Wizard: Select path options
The options are:

Do not remove PPPC path if pairs exists: If you choose this option, the path will not be
removed if there is a PPRC pair using this path.

Force removal of PPRC path even if pairs exists: This option enables you to force the
removal of existing PPRC paths even if there are PPRC pairs established that could use
this path. This option is helpful for the Failover/Failback Mode. For further information,
refer to 4.9, “PPRC connectivity” on page 101.
From the next panel, you can either Save, Run, or Cancel the task. As mentioned before,
once a task is saved, you can run it from the Tasks panel at any time.
Once you have run the task, all PPRC paths from the source to the target LSSs will be
removed and the blue asterisks below the Path Connection symbol in the Path Origin column
will disappear.
6.14 Establishing a Synchronous PPRC pair
Use the Volumes panel to establish PPRC pairs. On the left side, select the source LSS, and
on the right side, the target LSS. This is done using the drop-down menus at the top of the
Volumes menu.
The source and target logical subsystem are specified the following way: device type (4
digits): ESS Serial number (5 digits): LSS number (2 digits).
In our example shown in Figure 6-91 on page 267, we have selected 2105:18767:16 as the
source LSS and 2105:22331:13 as the LSS target of our PPRC pair. The volume 000 in LSS
16 (serial number of 60018767) is selected as a primary (source) volume. The volume 009 in
LSS 13 (Serial number of 30922331) is selected as a secondary (target) volume.
Tip: To remove a path, you can either use the Paths panel, the Tasks panel (by modifying
an Establish Paths task or by running a Remove Paths task), or by using the Logical
Subsystems panel. For further information, refer to the IBM TotalStorage Enterprise
Storage Server Web Interface User’s Guide, SC26-7448.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
267
Figure 6-91 Volumes panel: Selecting source and target volumes for PPRC
You always need to have two components to establish a PPRC pair, a primary volume (source
volume) and a secondary volume (target volume). Select the primary, and, with a right-click,
the secondary. If you have selected the wrong volumes, just click on the correct primary
volume again to clear the selection (Figure 6-91).
Once you have selected the primary and the secondary, do a second right-click on the
secondary (target) to bring up the Task Wizard (Figure 6-92). Select the Establish
Synchronous PPRC copy pair option and click Next.
Figure 6-92 Task Wizard: Select task type
Within the next panel, you can specify the copy options of the PPRC pair (Figure 6-93 on
page 268). Click Next when you have finished the selection.

268
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-93 Task Wizard: Select copy options
6.14.1 Options for Synchronous PPRC

Copy initialization:
– Do not copy volume: If this option is checked, the PPRC pair relationship is
established without copying any data from the source to the target. This option is used
when the source and target contain exactly the same data and are consistent while in
simplex state. Choosing this option causes the volumes to turn duplex immediately on
the assumption that both source and target contain identical data.
– Copy entire volume: If this option is checked, all data is copied from the source to the
target volume. This option has to be used the first time a PPRC relationship is going to
be established and is needed to guarantee that the source and the target contain the
same data.
– Copy out-of-sync cylinders only: This option copies only the data that was updated
on the source volume since a PPRC copy pair was suspended. The option is used to
resynchronize a Synchronous PPRC or a PPRC-XD pair.

PPRC modes:
– PPRC Failover: Select this option to reverse the direction of a PPRC pair. Use this
option to make your current secondary volume at the recovery site become a primary
volume (your current primary volume at the production side being the secondary
volume of the PPRC relationship). For further information, refer to 6.15.11, “Performing
a PPRC Failover” on page 293.
– PPRC Failback: Select this option to resynchronize the PPRC volumes at the recovery
site with their peer volumes at the production site. For further information, refer to
6.15.12, “Performing a PPRC Failback” on page 296.

Copy options:
– Permit read from secondary: Selecting this option allows host servers to read from
the PPRC secondary volume. The PPRC pair must be in a full-duplex state in order for
the host server to read the volume. This option is helpful for operating systems that use
a Logical Volume Manager that reads data structures from the header of a secondary

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
269
volume to update internal databases, such as the ODM under AIX, or /etc./lvmtab
under HP-UX. For further information, refer to Chapter 8, “Open systems specifics” on
page 407.
– Suspend PPRC after establish complete: If you select the Copy entire volume or
Copy out-of-sync cylinders only option described above, this means that after the
data is completely copied to the target volume, the target volume goes into the
suspended state. Selecting this option saves you the effort of creating a separate task
to suspend the pair. This is useful in a PPRC-XD catch-up operation over extended
distances.
– Asynchronous Cascading PPRC: Select this option to allow a PPRC secondary
volume to also be a PPRC primary volume for a different relationship. You can use this
option for creating a consistent copy of your data at a tertiary site. For further
information, refer to 6.16, “Establishing an Asynchronous Cascading PPRC” on
page 300.
From the next panel, you can either Save, Run, or Cancel the copy task, as shown in
Figure 6-94. Click Save to run the task and run it later. To run the task immediately, click Run.
Figure 6-94 Task Wizard: Define Task
Once a Synchronous PPRC pair is established, the display of the primary and secondary
volumes from the Volumes panel changes, indicating that the initial copy is in progress
(Figure 6-95 on page 270).
Tip: After you create and save the task to establish Synchronous PPRC copy pairs, we
recommend that you create the tasks that suspend the pairs and terminate the pairs. You
can save time in creating tasks if you use the task you just saved as a template to create
the suspend task. Then use the suspend task as a template to create the terminate task. In
the Task panel, select the task you created and click on the Modify button to change the
options of the task. Then, give it a new name and save it as a new task.

270
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-95 Two volumes in a PPRC relationship and in copy pending status
Select a volume and click the Information Panel button to retrieve more information about
the status. If the source of a PPRC pair is selected, the number of out-of-sync sectors that are
still left to copy are displayed. Those are the tracks that need to be copied from the source to
the target to achieve full copy mode (Figure 6-96).
Figure 6-96 Information panel on a PPRC volume

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
271
Once the volumes are in sync, the icons of the volumes icons will change to solid.
(Figure 6-97).
Figure 6-97 PPRC volume in full duplex mode
6.15 PPRC-XD
In this section, we discuss the procedures related to PPRC-XD.
6.15.1 Establish a PPRC Extended Distance copy pair
PPRC Extended Distance provides the following advantages over the standard Synchronous
PPRC mode:

Minimal response-time delays for application hosts that are writing to a PPRC primary
device.

Copy distances that can be increased to thousands of miles with channel extenders.

Data migration without the response-time penalty of the Synchronous PPRC copy mode.

No FlashCopy of the primary volume required to eliminate the synchronous-write penalty.
Refer to 4.3, “PPRC Extended Distance (PPRC-XD)” on page 61 to get more information
about how PPRC-XD works and what benefits you can get from it in your production
environment.
To establish the PPRC Extended Distance copy pair, use the Volumes panel. This procedure
is almost identical to the previous one, which you used to create a Synchronous PPRC copy
pair, except that you select the Establish Extended Distance PPRC copy pair option in the
Select task type panel of the Task Wizard (Figure 6-98 on page 272).
Tip: Alternatively, you can use the Logical Subsystem panel to select Establish
Synchronous PPRC Pairs for an entire LSS. For further information, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.

272
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-98 Establishing a PPRC-XD between two volumes
When the option is selected, click Next to select the options for this task (Figure 6-99).
Figure 6-99 Task Wizard: Select copy option
6.15.2 Copy options for PPRC-XD
Here is a short description of the available options for PPRC-XD. To get a more detailed
description of these options, refer to 4.3.2, “PPRC-XD tasks and options” on page 62.

Copy initialization:
– Copy entire volume: If this option is checked, the two volumes will be in a PPRC-XD
duplex pending mode, considering that all the data from the primary volume needs to

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
273
be sent to the secondary volume. This option is the default option. It will be used to
initiate the PPRC-XD relationship.
– Do not copy volume: If this option is checked, when the PPRC-XD pair relationship is
established, the primary and secondary volumes will be in the PPRC-XD duplex
pending mode, but the ESS will consider that no data needs to be sent to the
secondary. Only new updates to the primary volumes will be sent to the secondary
volume. This option is used when the source and target contain exactly the same data
and are consistent while in simplex state, or, when establishing an Asynchronous
Cascading PPRC to avoid transferring unnecessary data (see 6.16, “Establishing an
Asynchronous Cascading PPRC” on page 300).
– Copy out-of-sync cylinders only: This option copies only the data that was updated
on the target volume since a PPRC-XD copy pair was suspended. The option is used
to resynchronize a PPRC-XD pair.

PPRC modes:
– PPRC Failover: This option is used to suspend the pair and cause both sides to
become primary. This allows the secondary site to access the data on the secondary
volume.
– PPRC Failback: Select this option to resynchronize the PPRC volumes at the recovery
site with their peer volumes at the production site. For further information, refer to 9.3,
“Types of PPRC Failover and Failback procedures” on page 460.

Copy options:
– Permit read from secondary: Selecting this option allows host servers to read from
the PPRC secondary volume. The PPRC-XD pair does not need to be in a full-duplex
state in order for the host server to read the volume. This option is helpful for operating
systems that use a Logical Volume Manager that reads data structures from the
header of a secondary volume to update internal databases, such as the ODM under
AIX, or /etc/lvmtab under HP-UX. For further information, refer to Chapter 8, “Open
systems specifics” on page 407.
– Suspend PPRC after establish complete: This option is not available for PPRC
Extended Distance. The reason this requirement is not imposed is that the duplex state
is not enforced by PPRC Extended Distance.
– Asynchronous Cascading PPRC: Select this option to allow a PPRC secondary
volume to also be a PPRC primary volume for a different relationship. You can use this
option for creating a consistent copy of your data at a tertiary site.
From the next panel, you can either Save, Run, or Cancel the copy task, as shown in
Figure 6-100 on page 274. Click Save to run the task and run it later. To run the task
immediately, click Run.

274
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-100 Task Wizard: Define Task
Once a PPRC Extended Distance pair is established, the PPRC-XD icons will appear next to
the source and target volumes (Figure 6-101).
Figure 6-101 Volumes panel: PPRC-XD established between volumes 700 and 605
Tip: Alternatively, you can use the Logical Subsystem panel to select Extended Distance
PPRC for an entire LSS. For further information, refer to the IBM TotalStorage Enterprise
Storage Server Web Interface User’s Guide, SC26-7448.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
275
6.15.3 Converting a PPRC-XD copy pair to Synchronous PPRC
There are two common situations when you would convert a Peer-to-Peer Remote Copy
(PPRC) copy pair from PPRC Extended Distance (PPRC-XD) mode to Synchronous PPRC:

Situation 1: You have used PPRC-XD to complete the bulk transfer of data in the creation
of many copy pairs, and you now want to convert some or all of those pairs to
Synchronous PPRC mode.

Situation 2: You have PPRC-XD copy pairs for which you want to make FlashCopy
backups on the remote site. You convert the pairs temporarily to Synchronous PPRC
mode in order to obtain a point-in-time consistent copy.
You can convert a PPRC Extended Distance copy pair to a synchronous pair using the
Volumes panel. Select the LSS within which you want to convert the PPRC Extended
Distance copy pair to a synchronous pair. Click on the primary PPRC volume. In the Target
column of the Volumes panel, the secondary volume of the PPRC pair will be displayed
(Figure 6-102).
Once you have selected the source volume, right-click on the target volume, then right-click it
again to open the Task Wizard (Figure 6-102).
Figure 6-102 Volumes panel and Task Wizard
Select the Establish Synchronous PPRC copy pair option and click Next. On the next
panel, you can select the copy options (Figure 6-103 on page 276).

276
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-103 Task Wizard: Select copy options for PPRC-XD
Select Copy out-of-sync cylinders only option. Refer to 6.15.1, “Establish a PPRC
Extended Distance copy pair” on page 271 to get details about these options. Other options
can also be selected.
Figure 6-104 Task Wizard: Define Task
Click Next. From the next panel (Figure 6-104), you can either Save, Run, or Cancel the
task.
Once you run the task, all out-of-sync cylinders will be copied from the primary to the
secondary volume. Once the copy starts, the PPRC-XD icons will be removed and will
change to copy-pending icons. The volumes will now be in
Copy pending
mode in a
Synchronous PPRC relationship (Figure 6-105 on page 277).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
277
Figure 6-105 Volumes panel: Resynchronizing volumes
In our example, we selected the Suspend after establish option. So, at the end of the copy
of all out-of-synchronous tracks, when the two volumes are in
duplex
mode, they change
immediately to the
suspended
mode. The icons will automatically change to show the
suspended state when the copy of out-of-sync tracks is finished (Figure 6-106).
Figure 6-106 Volumes panel: Volumes are in suspended state after a fullcopy is established

278
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.15.4 Creating a backup copy of volumes via PPRC Extended Distance
When you have used the PPRC Extended Distance (PPRC-XD) mode to transfer data, and
the distance between the local ESS and the remote ESS is close enough for you to establish
a Synchronous PPRC copy pair that can achieve the full duplex state, you can temporarily
convert from PPRC Extended Distance mode to Synchronous PPRC mode when you want to
make a consistent copy of the volumes.
This process would be useful as a backup copy and for data migration purposes, but would
not be particularly useful for Disaster Recovery if the source data set is subject to rapid
change. For further information, refer to 4.3, “PPRC Extended Distance (PPRC-XD)” on
page 61.
Perform the following steps to obtain a backup point-in-time consistent copy of the PPRC-XD
volumes that are at the remote site:
1.Change the copy mode for the PPRC Extended Distance copy pair to Synchronous PPRC.
Select the Suspend after establish option in the Task Wizard in creating the
Synchronous PPRC. This avoids the step of creating a separate task to just suspend the
pairs (see 6.15.3, “Converting a PPRC-XD copy pair to Synchronous PPRC” on
page 275).
2.Alternatively, if you did not select the Suspend after establish option, suspend the
Synchronous PPRC copy pair after it reaches the full duplex state (consistency) (see
6.15.5, “Suspending a PPRC pair” on page 278).
3.Perform a FlashCopy on the secondary volumes (see 6.11.1, “Establishing a FlashCopy
pair” on page 226).
4.Resume PPRC Extended Distance mode for the copy pair. If you have saved the task that
created the original PPRC-XD relationship (see 6.15.1, “Establish a PPRC Extended
Distance copy pair” on page 271“), you can invoke the saved task from the Tasks panel.
6.15.5 Suspending a PPRC pair
When you suspend a copy pair, PPRC stops transferring data to the target volume. Because
the primary ESS keeps track of all changed cylinders on the source volume, you can resume
PPRC operations at a later time. To resume operations, run a task that resynchronizes the
PPRC copy pair (using the Copy out of sync cylinders only option) (see 6.14, “Establishing
a Synchronous PPRC pair” on page 266).
ESS Copy Services provides an alternative to creating a separate task that suspends a
PPRC pair. When you establish the pair, the Task Wizard provides the option Suspend PPRC
after Establish (for details, see 6.14, “Establishing a Synchronous PPRC pair” on page 266).
To suspend a PPRC copy pair, you use the Volumes panel. This procedure is almost identical
to the one described in 6.14, “Establishing a Synchronous PPRC pair” on page 266, except
that you select the Suspend PPRC copy pair option in the Select task type panel of the Task
Wizard (Figure 6-107 on page 279).
Tip: Alternatively, you can use the Logical Subsystem panel to select and convert PPRC
Extended Distance copy pairs to Synchronous PPRC. Running the tasks at the LSS level
will perform the task on several volumes. For further information, refer to the IBM
TotalStorage Enterprise Storage Server Web Interface User’s Guide, SC26-7448.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
279
Figure 6-107 Task Wizard: Selecting Suspend PPRC copy pair option
If you click Next within the next panel, you can decide whether to schedule the task with the
source volume or to schedule the task to the target volume (Figure 6-108).
Figure 6-108 Task Wizard
The task runs on either the source or target ESS. For example, if the source LSS is not
available, you can schedule a suspension of a PPRC pair with the target LSS. This is useful in
a disaster situation when the primary site has gone down. Refer to Chapter 9, “Disaster
Recovery” on page 453 to get more details.
The result of a suspend task with the Schedule with the source option selected is shown in
Figure 6-109 on page 280.
Note:

If you schedule the task with the source logical subsystem, the source will become
suspended. As a result, the target will become suspended immediately after
(Figure 6-109).

If you schedule the task with the target logical subsystem, the target will become
suspended, but the source might not become suspended. You should use this option
only in a disaster situation when the source LSS is not available anymore. (An example
of the result of this option is shown in Figure 6-110.)

280
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-109 Volumes panel: Result of suspend task with Schedule with the source option
The result of a suspend task with the Schedule with the target option selected is shown in
Figure 6-110.
Figure 6-110 Volumes panel: Result of suspend task with Schedule with the target option
6.15.6 Terminating a PPRC copy pair
When you terminate a PPRC copy pair, the PPRC relationship between the source and target
volume ends. The source and target volumes are removed from the PPRC configuration. You
can either use this function to terminate a Synchronous PPRC pair or an Extended Distance
PPRC pair.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
281
You can terminate a PPRC copy pair in the following ways:

By modifying a task used to establish or suspend a PPRC copy pair that has been saved
(refer to 6.8.5, “Modifying a task” on page 221 for further information)

By using the Volumes panel or the LSS panel
This procedure is almost identical to 6.14, “Establishing a Synchronous PPRC pair” on
page 266; you first select the primary (source) volume and the secondary (target) volume, as
shown in Figure 6-111.
Figure 6-111 Volumes panel: Two volumes in a PPRC relationship selected
To terminate the pair, you select the Terminate PPRC copy pair option in the Select task
type panel of the Task Wizard (Figure 6-112). Then click Next.
Figure 6-112 Task Wizard: Select task type
Within the next panel, you can decide whether to Schedule task with the source logical
subsystem or to Schedule task with the target logical subsystem (Figure 6-113 on
page 282).

282
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-113 Task Wizard: Specify logical subsystem to execute the task
The result of a terminate PPRC pair with the source Logical Subsystem selected is shown in
Figure 6-114. Note that the status icons have been removed.
Figure 6-114 Volumes panel: Result of a terminate PPRC with Schedule with target option
The result of a terminate PPRC pair with the target Logical Subsystem selected is shown in
Figure 6-115 on page 283.
Note:

If you schedule a Terminate PPRC pair task with the
source
logical subsystem, and
you run the task successfully, the source and the target volume go to a simplex state
(see Figure 6-114).

If you schedule a Terminate PPRC pair task with the
target
logical subsystem, and the
task has been completed successfully, the source volume is in a suspended state, and
the target volume is in the simplex state (see Figure 6-115). This option is useful in a
disaster situation when the primary site has gone down.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
283
Figure 6-115 Volumes panel: Result of a terminate PPRC with Schedule with target option
6.15.7 Resynchronizing PPRC copy pairs
When you resynchronize a copy pair, all changed cylinders on the source volume are copied
to the target volume. Synchronous PPRC operations are then resumed for the pair. You will
use this method to resynchronize two volumes in suspended mode. The method described
here is the same method as 6.15.3, “Converting a PPRC-XD copy pair to Synchronous
PPRC” on page 275.
Use the Volumes panel to resynchronize a PPRC copy pair. On the left side, select the source
LSS, and on the right side, the target LSS. This is done using the drop-down menu at the top
of the Volumes menu. Select the source volume, and, with a right-click, the target. Once you
have selected the source and the target, you do a second right-click on the target to bring up
the Task Wizard (Figure 6-116 on page 284).

284
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-116 Volumes panel and Task Wizard
Select the Establish Synchronous PPRC copy pair option and click Next. On the next
panel, you can select the copy options (Figure 6-117).
Figure 6-117 Task Wizard: Select copy options
Select the Copy out-of-sync cylinders only initialization option and decide whether to use
the following two options (those options are described in 6.14, “Establishing a Synchronous
PPRC pair” on page 266):

Permit read from secondary

Suspend PPRC after establish complete

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
285
Click Next. From the next panel (Figure 6-118), you can either Save, Run, or Cancel the
task. Once you run the task, all out-of-sync cylinders are copied from the primary to the
secondary volume.
Figure 6-118 Task Wizard: Define Task
6.15.8 Creating a PPRC Consistency Group
The PPRC Consistency Group will be used to get consistent data at the remote site when
there are unplanned outages and other problems. It can also be used to build consistent data
at a remote site for backup purposes, for example. Refer to 4.5, “Asynchronous PPRC” on
page 74 for more information.
You will define three different types of tasks related to Consistency Groups:
1.Tasks that Establish Path with the PPRC Consistency Group option. The implementation
of this task is explained in detail in this section.
2.Tasks that Freeze PPRC Consistency Group. These tasks are established at the LSS
level. The implementation of this task is explained in detail in 6.15.9, “Freezing a PPRC
Consistency Group” on page 289.
3.Tasks that resume the I/O on the
frozen
volumes (Consistency Created tasks). These
tasks are established at the LSS level. The implementation of this task is explained in
detail in 6.15.10, “Thawing a PPRC Consistency Group” on page 291.
Here is a short description of how and under which circumstances those tasks will be
executed. For more information, refer to 4.5, “Asynchronous PPRC” on page 74.
A PPRC Consistency Group is a set of PPRC volume pairs that have the same source and
target LSS. ESS Copy Services creates the Consistency Group when you run a task to
establish a path between the primary and secondary LSSs if this task has the PPRC
Consistency Group option selected.
Tip: Alternatively, you can use the Logical Subsystem Panel to select Resynchronize
PPRC Copy Pairs. For further information, refer to the IBM TotalStorage Enterprise
Storage Server Web Interface User’s Guide, SC26-7448.

286
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
When an error occurs that affects any of the volumes in a PPRC Consistency Group, the ESS
causes the volume where the error is detected to enter a suspended state. If the volume is
participating in a Consistency Group, it enters a
Queue Full
state. The primary host
temporarily queues all the updates to the primary volume. The primary volume will stay in the
Queue Full
state for two minutes (two minutes is the default value; you can modify this value
using the LSS Properties panel (see 6.6.4, “LSS properties” on page 212)).
Within those two minutes, you can detect that the volume enters a
Queue Full
state (via
SNMP traps, e-mail notification, or host I/O error messages) and trigger the execution of the
task that will freeze all the volumes in the PPRC Consistency Group. This task will remove the
established paths between the LSSs, causing the primary volumes to be in a suspended
state. Furthermore, if the volumes in the LSS are in a Consistency Group, all primary volumes
will be in a
Queue Full
state.
Once the Freeze command is executed, you will execute a Consistency created task to
resume the
Queue Full
state of the primary volume. This will allow the primary hosts to
perform write I/Os on the primary volumes. If you do not perform this Consistency Created
task, the
Queue Full
state will be automatically resumed after a delay (default is two minutes).
To come back to the initial state, you must reestablish the paths between the two LSSs (The
freeze command removes the paths between LSSs) and reestablish the PPRC relationship.
Establishing a path with the PPRC Consistency Group option
To create a PPRC Consistency Group, click Paths in the navigation frame of an ESS Copy
Services panel. The Paths panel opens.
From the Logical Subsystem Sources list, select the logical subsystem (LSS) from which you
want to create the paths. Click on the path in the Local Port/Remote Port column. Refer to
Figure 6-119 on page 287 as you go through the steps listed here:
1.Select Work with Paths.
2.Select the source ESS from the Logical Subsystem sources.
3.Click on the source LSS.
4.Select the target ESS in the drop-down menu from the Logical Subsystem targets.
5.Click on the target LSS.
6.Click on the desired path in the third column labeled Local Port/Remote Port.
7.Right-click on the highlighted path.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
287
Figure 6-119 Paths panel: Establishing a path
8.Select Establish path and click Next.
Figure 6-120 Task Wizard: Select path options - FCP link
9.Because we are working with an FCP link in our example, the Select path options Task
Wizard will be like the one shown in Figure 6-120. Select PPRC Consistency Group and
click Next. The Define task panel of the Task Wizard opens.
If you are working with an ESCON port, then you will see the Task Wizard shown in
Figure 6-121 on page 288.

288
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-121 Task Wizard: Select path options - ESCON link
You can either click Save to save the task and run it later, or click Run to run the task
immediately (see Figure 6-122).
Figure 6-122 Task Wizard: Define Task
Attention: When you create paths for PPRC Consistency Groups, it is important to save
the task. After you run the Freeze command and correct the problem that caused the
suspension, you need to reestablish the paths. You must reestablish the paths before you
run the task that resynchronizes the PPRC volumes.
Note: There is another way to create a Consistency Group: You can create a Consistency
Group, including all the volumes in one LSS, by clicking on the PPRC Consistency Group
check box in the LSS properties panel (see “PPRC Consistency Group” on page 212).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
289
6.15.9 Freezing a PPRC Consistency Group
This task will remove the established path between two LSSs, causing the primary volumes to
be in a suspended state. So performing a Freeze PPRC Consistency Group stops all
operations from being propagated to the secondary volumes in the LSS for all volumes that
were using this path. Furthermore, if any volumes in the LSS are in a Consistency Group, all
primary volumes will be in a
Queue Full
state.
When you want to freeze several Consistency Groups of volumes spread across different
LSSs, the freeze operation is best implemented by creating and saving tasks for all the
associated LSS pairs and then grouping the tasks. When you need to perform a freeze
operation, the group can be run from the Task panel or from the command line interface. See
6.8.2, “Grouping and ungrouping tasks” on page 219 for information on grouping tasks.
For the freeze operation to work and create a consistent image of the data at the remote site,
the Establish Paths task must indicate that the LSS to LSS relationship is part of a
Consistency Group (see 6.15.8, “Creating a PPRC Consistency Group” on page 285).
You can invoke the freeze task when a freeze trigger event is detected (for example, an e-mail
notification, an SNMP alert, or a host-detected failure condition). Following the freeze
operation, a
Consistency-Group-created
operation should be run to stop any
Queue Full

states that the volumes are in. See 4.5, “Asynchronous PPRC” on page 74 for more
information about the freeze and Consistency-Group-created operations.
To create a task that freezes a Consistency Group, click the Logical Subsystems button in
the navigation frame of an ESS Copy Service panel. The Logical Subsystems panel opens,
and within the Select list in the upper right corner of the panel, select the source ESS. A panel
showing all the logical subsystems (LSSs) is displayed, with the appropriate names and serial
numbers indicated below each LSS icon. Click the source LSS. The LSS ID of the source
LSS turns blue (Figure 6-123). Click the Select list again and select the target ESS. The LSSs
of this ESS are displayed; right-click on the target LSS. The LSS ID of the target LSS turns
red (Figure 6-124 on page 290).
Figure 6-123 LSS panel: LSS source selected

290
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-124 LSS panel: LSS target selected
Right-click again on the target LSS. A warning message may pop up if the configuration of the
selected source and target LSSs are different (see Figure 6-124). If so, click OK on this
message, and right-click again on the target LSS.
In the Task Wizard, select Freeze PPRC Consistency Group (Figure 6-125), and then click
Next.
Figure 6-125 Task Wizard: Select task type
Within the next panel (Figure 6-126 on page 291), you can decide whether to Run, Save, or
Cancel the task. If you want to save the task, type the task name (mandatory) and the task
description (optional) in the appropriate fields. You can also click Run to execute the task
immediately.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
291
Figure 6-126 Task Wizard: Define Task
6.15.10 Thawing a PPRC Consistency Group
The thawing of a PPRC Consistency Group (Consistency-Group-created operation) follows
the successful freeze for all LSSs in the Consistency Group. It allows normal I/O operations
from the host to the ESS to resume. For further information, refer to 4.5, “Asynchronous
PPRC” on page 74).
You can either use the Task panel to modify a Freeze PPRC Consistency Group

task to
create an appropriate task to thaw the Consistency Group (refer to 6.8.5, “Modifying a task”
on page 221 for further information), or you can use the Logical Subsystems panel to create a
task to thaw a PPRC Consistency Group.
This procedure of using the Logical Subsystems panel is almost identical to the previous
procedure, which you used to freeze a PPRC Consistency Group, except that you select the
Consistency Created option in the Select task type panel of the Task Wizard (Figure 6-127
on page 292).
Attention: After creating and saving the task to freeze a PPRC Consistency Group, you
should create the corresponding Consistency-Group-created task. We recommend that
you modify the task that you just created so that it runs a Consistency-Group-created
operation, then name and save the task. For further information about modifying a task,
refer to 6.8.5, “Modifying a task” on page 221.

292
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-127 Task Wizard: Select task type
Click Next. In the next panel (Figure 6-129), you can decide whether to Run, Save, or Cancel
the task. If you want to save the task, type the task name and the task description in the
appropriate fields. Click Run to execute the task immediately.
Figure 6-128 Task Wizard: Select copy options
Figure 6-129 Task Wizard: Define Task

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
293
6.15.11 Performing a PPRC Failover
The PPRC Failover task can be used in a disaster recovery process. This task is part of the
process used in Chapter 9, “Disaster Recovery” on page 453 to restart a production
environment using the volumes from the recovery site. In a disaster recovery process, the
failover task is followed by a failback task once a path from the recovery site to the production
site is established.
Use the Volumes panel to establish the PPRC pairs. The process is identical to the one used
when establishing a PPRC relationship. Refer to 4.2, “Synchronous PPRC” on page 57 to get
a full detailed description of the process.
When creating the PPRC Failover task, you must be very careful when choosing the source
volume and the target volumes for the task:

The secondary volume of the established PPRC pair should be selected as the primary
(source) volume for the new PPRC Failover task. Click to select the volume.

The primary of the established PPRC pair should be selected as the secondary (target)
volume for the new PPRC Failover task. Right-click to select the volume.
Figure 6-130 shows an example of how the volumes must be selected:

A PPRC relationship is established and running between the primary volume 60018767 at
the production site and the secondary volume 30922331 at the recovery site.

The volume 30922331 is the secondary (target) volume for the established PPRC, so it is
selected as the primary (source) volume.

The volume 60018767 is the primary (source) volume for the established PPRC, so it is
selected as the secondary (target) volume.
Figure 6-130 Volumes panel: Volume selection for a PPRC Failover
Once the two volumes are selected, another right-click on the target volume will open the
Task Wizard.

294
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
In the Task Wizard, select the Establish Synchronous PPRC copy pair task (Figure 6-131)
and click Next.
Figure 6-131 Task Wizard: Select task type
In the second Task Wizard panel (Figure 6-132), select the PPRC Failover option and click
Next.
Figure 6-132 Task Wizard: Select copy options
In the next panel (Figure 6-133 on page 295), you can decide whether to Run, Save, or
Cancel the task. If you want to save the task, type the task name and the task description in
the appropriate fields. You can also click Run to execute the task immediately.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
295
Figure 6-133 Task Wizard: Define Task
Once the PPRC Failover task is established on a Synchronous PPRC pair, the volumes states
change, as shown in Figure 6-134 on page 296:

The initial primary volume at the production site remains unchanged. Its state is: Source
volume of a PPRC relationship. The target volume of this relationship is the volume at the
recovery site. The status of the volume at the production site is: Full duplex of
Synchronous PPRC relationship. If you perform any write operation on this volume, its
status will change from Full duplex to Suspended.

The initial secondary volume at the recovery site changed from target volume to the new
state of: Source volume of PPRC relationship. The target of this relationship is the initial
primary volume at the local site. The status of the volume at the recovery site is:
Suspended of Synchronous PPRC relationship.
After the PPRC Failover task, the volume at the recovery site is available to be mounted and
used by a host. You can use this volume to restart your application from the recovery site.

296
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-134 Volumes panel: Result of a failover task
Based on the volume IDs used in the screen shots, Table 6-4 presents a summary of the
PPRC Failover rules when selecting the volumes and the result of the volume status after
running the task.
Table 6-4 Summary of the PPRC Failover creation and results
6.15.12 Performing a PPRC Failback
This task is part of the process used to restart a production environment using the volumes
from the recovery site (see Chapter 9, “Disaster Recovery” on page 453 for details). The
failback task is generally performed after a failover task.
After establishing a PPRC Failover task, the volume at the recovery site is available to be
mounted and used by a host. You can use this volume to restart your application from the
recovery site (see previous section). Establishing PPRC with the Failback option will
Production Site
Volume 60018767
Recovery Site
Volume 30922331
Initial status, before
running the PPRC
Failover task
Primary volume (source) of a
PPRC.
Full Duplex.
The target (secondary) volume is
volume 30922331.
Secondary volume (target) of the
PPRC
Full Duplex.
The source (primary) volume is
volume 60018767.
Selection rules, when
creating the PPRC
Failover task
Select as secondary (target)
volume.
Select as primary (source) volume.
Final status, after
running the PPRC
Failover task
Primary volume (source) of a
PPRC.
Full Duplex.
The target (secondary) volume is
volume 30922331.
Primary volume (source) of a
PPRC.
Suspended.
The target (secondary) volume is
volume 60018767.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
297
resynchronize the PPRC volumes at the recovery site with their peer volumes at the
production site.
Use the Volumes panel to establish the PPRC pairs. The process is identical to the one used
when establishing a PPRC relationship. Refer to 6.14, “Establishing a Synchronous PPRC
pair” on page 266 to get a full detailed description of the process.
When creating the PPRC Failback task, you must be very careful when choosing the source
volume and the target volumes for the task.

The primary (source) volume of your new PPRC Failback task must be the volume at the
recovery site. This volume was initially the secondary volume of the PPRC relationship.
After the PPRC Failover task has run, this volume state is changed to become a primary
(source) volume in a Suspended state. Click to select the volume.

The secondary (target) volume of your new PPRC Failback task must be the volume at the
production site. This volume is the primary volume of the initial PPRC relationship. After
the PPRC Failover task has run, this volume state remains unchanged. If you perform any
write operation to this volume after the PPRC Failover task was established, then its status
changes to Suspended. Right-click to select the volume.
Figure 6-135 shows an example of how the volumes must be selected:

A PPRC Failover task was performed on an established PPRC pair (primary volume was
60018767 at the production site and the secondary volume was 30922331 at the recovery
site).

The volume 30922331 at the recovery site is now a primary volume in a PPRC
relationship and is in a suspended state, so it is selected as the primary (source) volume.

The volume 60018767 at the production site is still the primary (source) volume for the
established PPRC, so it is selected as the secondary (target) volume.
Figure 6-135 Volumes panel: Selecting the volume for a PPRC Failback task
Important: Before you run the PPRC Failback task, it is necessary to establish a PPRC
path from the recovery site to the production site between the concerned LSSs.

298
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Once the two volumes are selected, another right-click on the target volume will open the
Task Wizard (Table 6-136).
In the Task Wizard, select the Establish Synchronous PPRC copy pair task (Figure 6-136)
and click Next.
Figure 6-136 Task Wizard: Select task type
In the second Task Wizard panel (Figure 6-137), select the PPRC Failback option and click
Next.
Figure 6-137 Task Wizard: Select copy options
In the next panel, you can decide whether to Run, Save, or Cancel the task. If you want to
save the task, type the task name and the task description in the appropriate fields. Click Run
to execute the task immediately.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
299
Figure 6-138 Task Wizard: Define Task
Once the PPRC Failback task is established, the PPRC relationship is resynchronized
between the recovery site and the production site. The volumes’ state changes are shown in
Figure 6-139:

The volume at the production site is now the secondary (target) volume of the PPRC
relationship.

The volume at the recovery site is now the primary (source) volume of the PPRC.

The status of the two volumes will be first Copy pending, then Full Duplex, when the copy
of the out-of-sync sectors is finished.
Figure 6-139 Volumes panel: Result of a failback task

300
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Based on the volumes IDs used in the screen shots, Table 6-5 presents a summary of the
PPRC Failover and PPRC Failback rules when selecting the volumes and the result on the
volume status after running each tasks.
Table 6-5 Summary of the PPRC Failover and Failback creation and results
6.16 Establishing an Asynchronous Cascading PPRC
PPRC Version 2 includes a new option, Asynchronous Cascading PPRC, when establishing a
PPRC relationship. This option allows a PPRC secondary volume to also be a PPRC primary
volume for a different relationship.
You will use this option to create an Asynchronous Cascading PPRC relationship across
three ESS volumes (the primary volume, the secondary volume and the tertiary volume). To
get details about the benefits of Asynchronous Cascading PPRC, refer to 4.4, “Asynchronous
Cascading PPRC” on page 67.
6.16.1 Terminology
Here is the terminology that is used when discussing Asynchronous Cascading PPRC:

A site where the production applications run is referred to as the
local site
or the
application site
.

An
intermediate site
is also called a
bunker site
and represents a site where the
application site data is mirrored by means of Synchronous PPRC. The local site and the
intermediate site are often two different locations, but they can also be the same location.

A site where the local site data is mirrored via the intermediate site is referred to as the
remote
site.
Production Site
Volume 60018767
Recovery Site
Volume 30922331
Initial status, before
running the PPRC
Failover task
Primary volume (source) of a
PPRC.
Full Duplex.
The target (secondary) volume is
volume 30922331.
Secondary volume (target) of the
PPRC
Full Duplex.
The source (primary) volume is
volume 60018767.
Selection rules, when
creating the PPRC
Failover task
Select as secondary (target)
volume.
Select as primary (source) volume.
Status, after running
the PPRC Failover task
Primary volume (source) of a
PPRC.
Full Duplex.
The target (secondary) volume is
volume 30922331.
Primary volume (source) of a
PPRC.
Suspended.
The target (secondary) volume is
volume 60018767.
Selection rules, when
creating the PPRC
Failback task
Select as secondary (target)
volume.
Select as primary (source) volume.
Final status, after
running the PPRC
Failback task
Secondary volume (target) of the
PPRC.
Copy pending.
The source (primary) volume is
volume 30922331.
Primary volume (source) of a
PPRC.
Copy pending.
The target (secondary) volume is
volume 60018767.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
301

The Synchronous PPRC pairs established between the volumes at the local host site and
the intermediate site are called the
local PPRC pairs
.

The PPRC pairs created between the intermediate site and the remote site are called the
remote PPRC pairs
.
6.16.2 Necessary tasks
In order to create an Asynchronous Cascading PPRC between three volumes, you have to
create and run four tasks:

Two tasks to establish the required paths:
a.One path is needed from the local to the intermediate site.
b.One path is needed from the intermediate path to the remote site.

One task to establish a PPRC relationship between the primary volume and the secondary
volume.

One task to establish a PPRC relationship between the secondary volume and the tertiary
volume. You must select the Asynchronous Cascading PPRC option for this task in order
to allow the secondary Volume of your Asynchronous Cascading PPRC to be both a
secondary volume and a primary volume in two different relationships.
6.16.3 Allowed configurations
There are three available configurations in Asynchronous Cascading PPRC:
1.SYNC

XD
This is a typical Asynchronous Cascading PPRC configuration. There is a Synchronous
PPRC relationship from the local site to the intermediate site, and then an asynchronous
PPRC-XD relationship from the intermediate site to the remote site.
In case of an unplanned outage at the local site, the intermediate site still contains all of
the data, since it is an exact copy of local site. The intermediate site would complete
sending the updates to the remote site and then the remote site would have a copy of data
up to the time of failure.
2.XD

XD
This configuration consists of a PPRC-XD relationship from the local site to the
intermediate site, and PPRC-XD relationship from the intermediate site to the remote site.
It may be used to make multiple copies of data, without impact on the application I/O. The
intermediate and remote volumes are
fuzzy
, and it is
necessary
, from time to time, to
resynchronize them in order to make point-in-time consistent backups.
3.SYNC

SYNC
This allows a Synchronous PPRC relationship between the local and intermediate sites as
well as between the intermediate and remote sites. However, this Asynchronous
Cascading PPRC configuration may have a severe impact on the application I/O, because
each write I/O has to wait for two PPRC data transfers. The application has to wait for the
confirmation that data is written to the intermediate and remote ESS cache and NVS.
This configuration may be useful to get a consistent copy at the remote location, but only if
there is little or no I/O at the local site.

SYNC

XD and SYNC

SYNC combination
This configuration combines the above described SYNC

XD and SYNC

SYNC PPRC
cascading solutions. Basically, the SYNC

XD configuration is used, but periodically you
can switch to SYNC

SYNC mode in order to force the remote PPRC pairs to go to

302
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
SYNC and take a consistent copy at the remote site. Switching remote PPRC pairs from
PPRC-XD to PPRC SYNC mode is usually done when the application using the primary
PPRC volume(s) is stopped or quiesced so there is no I/O at the local site.
6.16.4 Establishment order
There are basically two methods used to set up an Asynchronous Cascading PPRC
environment, and both of them are valid:

Recommended order
– Establish remote PPRC pairs in XD mode with the Do not copy volume option.
– Establish local PPRC pairs in XD mode with the Copy entire volume option and then
go to synchronous.

Alternative order
– Establish local PPRC pairs with the Copy entire volume option.
– Wait for the volumes to reach DUPLEX state.
– Establish remote PPRC pairs in XD mode with the Copy entire volume option.
6.16.5 Options to make a volume eligible for cascading
A pair can be made eligible to be a cascaded pair at the time the pair is established or at a
later time:

When a PPRC pair is initially established, the user can indicate that the pair is eligible for
cascading.

For an already established pair, the user can issue an establish command that makes it a
cascaded pair without breaking the existing pair to do the reestablish.
Some examples are listed below:

Intermediate Site

Remote site was already established with the indication that the
volume at the Intermediate site is eligible for cascading. When Local Site

Intermediate
site is established, this now becomes an Asynchronous Cascading PPRC relationship.

Intermediate Site

Remote Site was already established without the indication that it is
eligible for cascading. An establish is issued again from the volume at the Intermediate
site to the volume at the Remote site, but this time with the Asynchronous Cascading
PPRC option selected, and the pair remains established. Then Local Site

Intermediate
site can be established, without breaking the Intermediate Site

Remote Site pair. This
now becomes an Asynchronous Cascading PPRC relationship.

Local site

Intermediate Site is already established. Then Intermediate Site

Remote
Site can be established with an indication that the volume at the Intermediate Site is
eligible for cascading, without breaking the Local site

Intermediate Site pair. This now
becomes an Asynchronous Cascading PPRC relationship.
6.16.6 Establishing an Asynchronous Cascading PPRC with the WUI
You can invoke the PPRC cascading function using the ESS Copy Services Web User
Interface. More detailed descriptions of the ESS Copy Services Web User Interface are in
Note: It is possible to do other combinations, for example, in the first case above, the
remote pairs do not have to be established with the Do not copy volume option, but it is a
waste of bandwidth to do so. However, the options listed above may be the most efficient.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
303
Chapter 5, “ESS Copy Services Web User Interface prior to LIC 2.2.0” on page 123. In
addition, when you are planning to use the ESS Copy Services WUI for PPRC operation,
refer to the IBM TotalStorage Enterprise Storge Server Web Interface User’s Guide,
SC26-7448.
An Asynchronous Cascading PPRC environment consists of local, intermediate, and remote
sites. The following procedure should be applied when establishing an Asynchronous
Cascading PPRC environment (the example uses ESCON links and an ESS with LIC level
2.2.0):
1.Create the PPRC logical paths between the local and intermediate sites. As illustrated in
Figure 6-140, the source LSS 2105:18767:16 is selected, where 18767 is the ESS serial
number and the LSS number is 16 and represents the local site. A path is established to
the target LSS 2105:18767:17.
Figure 6-140 PPRC establish path: Local to intermediate site (ESS at LIC level 2.2.0)
In this example, the local and intermediate volumes are placed in the same ESS disk
subsystem but in different LSSs: LSS 17 is the intermediate. The path origin selected is
the ESS ESCON adapter with the SAID 0028 (see 6.7.4, “Display Connection Paths” on
page 216).
Note: The Paths panel that you see in Figure 6-140 and Figure 6-141 correspond to
ESSs with LIC level 2.2.0. This panel looks different when working with LIC level 2.3.0.
Refer to 6.13.1, “Establishing paths” on page 260 for a description and discussion of
the Paths panel when using LIC level 2.3.0.

304
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
2.Establish the PPRC paths between the intermediate and remote sites, as illustrated in
Figure 6-141.
Figure 6-141 PPRC establish path: Intermediate to remote site (ESS at LIC level 2.2.0)
As you can see in the Information Panel in Figure 6-141, the remote site LSS is number 16
and belongs to another ESS disk subsystem with serial number 22331.
3.Once the logical paths are established, create the PPRC-XD asynchronous relationship
between the volumes that reside at the intermediate site with the volumes at the remote
site. Select the appropriate remote PPRC primary and secondary (with a right-click)
volumes, launch the Task Wizard (with another right-click), and then select Establish
PPRC Extended Distance copy pair, as shown in Figure 6-142 on page 305.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
305
Figure 6-142 Establish PPRC-XD volume pair: Intermediate to remote site volume
In the above example, the remote PPRC pair consists of the intermediate volume
70018767 in LSS 17 in ESS 18767 (defined as an intermediate PPRC primary), and
remote volume 60522331 in LSS 16, but in ESS 22331 (defined as a remote PPRC
secondary).
In the Select copy options panel, select Do not copy volume as a copy initialization mode
and Asynchronous Cascading PPRC as a copy option (Figure 6-143).
Figure 6-143 Select copy option panel: Asynchronous Cascading PPRC

306
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The Asynchronous Cascading PPRC copy option allows a PPRC secondary volume to
also be a PPRC primary volume for a different PPRC relationship. This means that the
primary PPRC volume in the above defined task is eligible to be a target PPRC volume in
a local PPRC pair relationship, from the local site to intermediate site.
When the options are selected, click on Next. In the Define task panel (Figure 6-144),
decide whether you want to Save, Run, or Cancel the task.
Figure 6-144 Task Wizard: Define Task
4.To check if the intermediate volume is eligible for cascading, select the volume and click
on the Information Panel button at the left bottom of the Volumes panel. A Volume
Information panel pops up (see Figure 6-145 on page 307).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
307
Figure 6-145 Volume Information Panel
Among the other volume information, you can see if the cascading function for the
selected volume is allowed (see PPRC status in Figure 6-145).
5.As soon as the remote PPRC pair is established, create the Synchronous PPRC
relationship between the volume that resides at the local site with the volume at the
intermediate site. Select the appropriate PPRC primary and secondary volumes with a
right-click, and then open the Task Wizard with another right-click. Then select Establish
Synchronous PPRC copy pair, as shown in Figure 6-146 on page 308.

308
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-146 Intermediate volume as a PPRC primary and PPRC secondary
In this example, the local PPRC pair consists of the local and intermediate volumes in the
same ESS (18767) but in different LSSs. The local PPRC primary volume is in LSS 0A,
and the local PPRC secondary volume is in LSS 04.
In the Select copy options panel, select Copy entire volume as a copy initialization mode,
as shown in Figure 6-147.
Figure 6-147 Select copy option panel: Copy entire volume
When the option is selected, click on Next.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
309
In the Define Task panel (Figure 6-148), decide whether you want to Save, Run, or
Cancel the task.
Figure 6-148 Define Task Wizard
6.Once the PPRC local pair is established, both the primary and secondary volumes are in a
PPRC relationship, and in the copy pending state mode. The intermediate volume (ESS
18767, LSS 17, volume 70018767) has an associated icon combination, as shown in
Figure 6-149 on page 310.

310
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-149 Asynchronous Cascading PPRC: Local pair
In the example, the icons on the volume 70018767 indicate that the very same volume is
in a PPRC-XD relationship as the primary volume of a remote PPRC pair (blue icon) and
at the same time in a PPRC SYNC relationship as a secondary volume of the local PPRC
pair, in copy pending mode (red icon volume, half empty). This is the typical Asynchronous
Cascading PPRC configuration.
7.When the synchronization of the two volumes in the PPRC sync pair is complete, the icons
will automatically change to show the duplex state of the volumes, as in Figure 6-150 on
page 311.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
311
Figure 6-150 Asynchronous Cascading PPRC
6.16.7 PPRC Failover/Failback with Asynchronous Cascading PPRC
In this section, we present an example for setting up an Asynchronous Cascading PPRC
environment and how to use Failover/Failback to switch application I/O from the local site (site
A) to the intermediate site (site B) and back. This example is a detailed step-by-step
procedure that follows the discussion in 9.6, “Failover/Failback with Asynchronous Cascading
PPRC” on page 483. Failover and Failback for Asynchronous Cascading PPRC was included
in ESS LIC 2.3.x.
For the procedure, we use the ESS Copy Services Web User Interface panels to:
1.Set up the Asynchronous Cascading PPRC environment using Fibre Channel links
2.Fail over the workload to site B
3.Fail back the workload to site A
Configuration setup and procedure steps
The following is the configuration used in our example:

ESS A/B - Serial 24663, WWNN - 5005076300C09DEF

ESS C - Serial 23953, WWNN - 5005076300C09B29
Figure 6-151 on page 312 illustrates the configuration used for this example.
We have a Fibre Channel link from SAID 000C on ESS-A/B to SAID 002C on the same
ESS-A/B. This link is used to establish a PPRC path from LSS16 to LSS17 in ESS-A/B. This
is used for the Synchronous PPRC volumes.
We also have a Fibre Channel link from SAID 0008 on ESS-A/B to SAID 008C on ESS-C.
This link is used to establish a PPRC path from LSS17 on ESS-A/B to LSS16 on ESS-C. This
is used for our remote PPRC-XD volumes.

312
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The configuration in our example is a two site Asynchronous Cascading PPRC configuration,
as discussed in 4.4.3, “Asynchronous Cascading PPRC mode combinations” on page 70,
where site A and site B volumes reside on the same physical ESS (designated as ESS-A/B),
while site C volumes reside at a remote ESS (designated as ESS-C).
In our example, we use one volume in each LSS. The volume in LSS16 on ESS-A/B is
referred to as the site A volume, the volume in LSS17 on ESS-A/B is referred to as the site B
volume, and the volume in LSS16 on ESS-C is referred to as the site C volume.
Figure 6-151 Asynchronous Cascading PPRC configuration used in this example
The procedural steps are the following:

Establish the local and remote paths.

Establish the remote and local PPRC volume pairs.

PPRC Failover task to site B volumes.

Suspend the B

C PPRC sessions.

Establish PPRC paths B

A.

PPRC Failback task on site B volumes.

Establish PPRC paths A

B.

PPRC Failover task to site A volumes.

PPRC Failback to site A volumes.

Reestablish B

C sessions.
Each of these steps is discussed in detail in the following sections.
Establishing the local and remote paths
The first step for setting up the Asynchronous Cascading PPRC environment is to establish
the paths from LSS16 to LSS17 in ESS A/B (local path, site A to site B) and from LSS17 in
ESS-A/B to LSS16 in ESS-C (remote path, site B to site C). Figure 6-152 on page 313 shows
the Paths panel once the local path is defined (note the blue asterisks below the link between
LSS16 to LSS17 within ESS A/B).
LSS

17
LSS
16
LSS
16
A
volume
B
volume
C
volume
ESS A/B sn24663
ESS C sn23953
XD
SYNC
002C
000C
0008 008C

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
313
Figure 6-152 Path successfully defined between site A and site B LSSs
Figure 6-153 shows the Paths panel once the remote path is defined (note the blue asterisks
below the link between LSS17 in ESS-A/B to LSS16 within ESS-C).
Figure 6-153 Path successfully defined between site B and site C LSSs

314
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Establishing the remote and local PPRC volume pairs
Now we establish the PPRC remote volume pairs (one volume pair in our example) from
LSS17 on ESS-A/B (intermediate site, primary XD volume) and LSS16 on ESS-C (remote
site, secondary XD volume). This is our remote PPRC-XD pair, and Figure 6-154 shows the
options chosen when establishing the pair.
Figure 6-154 Options to create the XD pair with
Figure 6-155 shows the remote pair, once it has been established, as displayed in the
Volumes panel of the ESS Copy Services Web User Interface.
Figure 6-155 Remote PPRC-XD volume pair established

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
315
We then establish the PPRC local volume pairs (one pair in our example) from LSS16 on
ESS-A/B (local site, primary PPRC-SYNC volume) and LSS17, also on ESS-A/B
(intermediate site, secondary PPRC-SYNC volume). This is our local PPRC-SYNC pair, and
Figure 6-156 shows the options chosen when establishing this PPRC volume pair.
Figure 6-156 Options to create the local PPRC-SYNC pair
Figure 6-157 on page 316 shows the local PPRC-SYNC pair when it is in the process of initial
synchronization (duplex pending).

316
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-157 Local PPRC-SYNC pair during initial synchronization (duplex pending)
Once the local pair reaches full synchronization (duplex state), then the Asynchronous
Cascading PPRC configuration is set up and running.
PPRC Failover task to site B volumes
With the Asynchronous Cascading PPRC configuration setup and running, now we can start
the failover procedure at any moment, so we can stop application updates on site A and
resume them on site B volumes. This will leave site A available for any required maintenance
while applications run using site B volumes.
For this, we run a PPRC Failover task onto site B volumes. Figure 6-158 on page 317 shows
the options selected for this task.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
317
Figure 6-158 Doing a PPRC Failover on site B volumes
Figure 6-159 illustrates the state of the local pair of volumes (site A

site B) once the PPRC
Failover is completed.
Figure 6-159 PPRC Failover to site B completed

318
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
At this point, application I/O can be swapped to site B volumes.
Figure 6-160 shows the Information Panel for the intermediate volume of our example (the
volume in LSS17 on ESS-A/B); note the
Secondary Host Writes Enabled
status. This allows a
volume that is a PPRC target to accept host I/O.
Figure 6-160 Information Panel for site B volume after failover
In these conditions, the production applications can run using site B volumes. Site B volumes
are still secondary volumes for the local PPRC-SYNC relationship and at the same time are
primary for the remote PPRC-XD relationship. And because the
Secondary Host Writes

Enabled
status is on, they can receive application updates, even if they are secondary
volumes.
Once the required activities are completed on site A, we can resume application processing
back on site A volumes. The following steps discuss the procedure to switch back to site A
(Failback to site A).
Suspend the B -> C PPRC sessions
This is the first step in moving application I/O back to site A. The B

C PPRC-XD sessions
(one session in our example) must be suspended, as we need to establish a B

A PPRC
session in order to fail back, and B volumes cannot be the source of two sessions at the same
time.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
319
Figure 6-161 Display of remote PPRC-XD pair after suspending
Figure 6-161 shows the status of the PPRC-XD remote (site B <--> site C) volume pair after
suspending. The status of the PPRC-SYNC source volume in LSS16 on ESS-A/B (site A, not
shown in the Figure 6-161) has not changed.
Establish PPRC paths B -> A
We now establish the site B

site A PPRC paths (one in our example) to allow the site A
volumes to be synchronized with the site B volumes. Figure 6-162 shows the path once it has
been established, from site B to site A in preparation for the PPRC Failback task.
Figure 6-162 Site B to site A path established
PPRC Failback task on site B volumes
The PPRC Failback processing will synchronize the site A volumes with the site B volumes
(one volume pair in our example). It will also set the
trusted resync
flag to allow a later

320
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
re-synchronization of site C volumes with site B volumes, as these pairs currently remain
suspended.
Figure 6-163 shows (in the background) the state of the local volume pair before the PPRC
Failback operation and it also shows (in the foreground) the Task Wizard options selected for
the Failback task (note that in Figure 6-163 the site A volume is on the right side of the
Volumes panel, and the site B volume is on the left side of the panel).
Figure 6-163 PPRC Failback on site B volume: Task options
Figure 6-164 on page 321 shows the status of the local PPRC volume pair (site B volume on
the left side of the Volumes panel, site A volume on the right) while the PPRC Failback task is
progressing. Notice that the volumes are re-synchronizing (disk icons half shaded). Also
notice the
Trusted Primed for Resynchronization
icon on site B volume; this will allow the later
re-synchronization of the remote PPRC-XD pair (site B to site C).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
321
Figure 6-164 Volumes in LSS17 & LSS16 on ESSA while PPRC Failback is proceeding
Figure 6-165 shows the volumes state once the PPRC Failback has completed; the volume
pairs are now synchronized (disk icons fully shaded).
Figure 6-165 Volumes in LSS17 & LSS16 on ESSA after PPRC Failback on B has completed
Figure 6-166 on page 322 shows the information panel for the site B volume (in LSS17 on
ESS-A/B); note the
Trusted Primed for Resynchronization
flag.

322
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-166 Information Panel for site B volume after PPRC Failback
Site A volumes are now synchronized with site B volumes. We are ready to stop application
I/O processing on site B and switch back to site A. The following sections describe the steps
needed to switch back to site A.
Establish PPRC paths A -> B
We can now establish the PPRC paths from site A

site B (one path in our example).
Figure 6-167 on page 323 shows the path successfully established (blue asterisks) from site
A LSS 16 to site B LSS 17 (both in the same ESS-A/B).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
323
Figure 6-167 Path successfully established from site A LSS16 to site B LSS17
PPRC Failover task to site A volumes
We can execute the PPRC Failover task to the site A volumes (one volume pair in our
example). This will terminate the PPRC B

A relationship and establish the A

B
relationship.
Figure 6-168 shows the state of the volume pair before the PPRC Failover task is executed.
Figure 6-168 Local PPRC volume pair: State before PPRC Failover
Figure 6-169 on page 324 shows the state of the local volume pair after the PPRC Failover
task has completed.

324
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-169 Local PPRC volume pair: State after PPRC Failover task on volume A
PPRC Failback to site A volumes
The PPRC Failback task on site A volumes will complete the restore of the A

B PPRC
session and reset the
Enable Secondary Host Writes
flag on the B volumes. Figure 6-170
shows the volume pair before the task is initiated (background) and the copy options used for
the task.
Figure 6-170 PPRC Failback task: Copy options
Figure 6-171 on page 325 shows the state of the volumes after the PPRC Failback task to site
A volumes is completed. Site B volume is now the PPRC secondary (red disk icon).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
325
Figure 6-171 PPRC Failback task completed
Figure 6-172 shows the information Panel for the volume in LSS17 on ESS-A/B (the site B
volume). Notice that the
Trusted Primed for Resynchronization
flag is still set, but the “Enable
Secondary Host Writes” has been reset.
Figure 6-172 Information Panel for the site B volume after Failback to site A

326
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Reestablish B -> C sessions
Now that the site A

site B PPRC-SYNC session has been restored, we can reestablish the
site B

site C PPRC-XD pairs using the out-of-sync option. This resynchronization is
allowed, as the site B volumes have the “Trusted Primed for Resynchronization” flag set.
Figure 6-173 shows the remote volume pair (site B and site C volumes) before the
resynchronization task.
Figure 6-173 Remote volume pair before resynchronization (Trusted Primed for Resynch)
We reestablish the PPRC-XD relationship between the B and C volumes, using the options
shown in Figure 6-174.
Figure 6-174 Re-synchronization between B and C volumes
Once the resynchronization task completes, the “Trusted Primed for Resynchronization” flag
is reset. Figure 6-175 on page 327 shows the information panel for the site C volume once
the re-synchronization has completed.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
327
Figure 6-175 Information Panel for the site C volume after resynchronization
The completion of this last step restores the Asynchronous Cascading PPRC configuration to
its fully operative condition.
6.17 Managing Asynchronous PPRC
Asynchronous PPRC provides a long distance copy solution based on non-Synchronous
PPRC-XD technology, which also gives recoverable point-in-time consistency at the remote
site with no I/O performance degradation. Repeated consistent copies of all the volumes in
the Asynchronous PPRC session are taken with only a short delay between them. This delay
can be set by the user, but generally will be less than 10 seconds. The currency of the data at
the remote site is no more than the time since the last copy was taken, but more importantly,
because the volumes are all at the same point-in-time, consistency is maintained and
recovery is faster and with less data loss.
This is discussed in detail in Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on page 55. In
this section, we will describe the steps to use and manage Asynchronous PPRC using the
Web User Interface provided with the ESS.
The following section is broken up into major steps as follows:

Prerequisites. Some important information regarding the way the steps will be presented
and also what you need to successfully implement Asynchronous PPRC (see 6.17.1,
“Before you start” on page 328).

Configuration. A discussion on some of the common configurations Asynchronous PPRC
will be used in and also the configuration used in the steps that are following (see 6.17.2,
“Configuration” on page 329).

328
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

How to set up Asynchronous PPRC. Step by step procedures showing how to create the
environment, set up the volume relationships, and build the session (see 6.17.3, “Setting
up Asynchronous PPRC” on page 330).

The next section shows you how to start, monitor, and manage the Asynchronous session
(see 6.17.4, “Starting Asynchronous PPRC” on page 344).

How to pause or terminate the session (see 6.17.5, “Pausing and terminating
Asynchronous PPRC” on page 349).

Procedures to fail over to the remote site after an outage or site disaster (see 6.17.6,
“Switching sites after a disaster” on page 351).

How to ensure the consistency of the data at the remote site, then recover the volumes
ready for operations and applications systems recovery (see 6.17.7, “Remote site
recovery” on page 354).

We demonstrate how Asynchronous PPRC can be used to take point-in-time copies of
data at the remote site for testing, quality assurance, or point-in-time backups without
disruption to the primary site; this is a planned copy (see 6.17.8, “Planned copy at remote
site” on page 363).

Once running backup systems at the remote site, the data will, at some time in the future
after the disaster is resolved, need to be copied back to the production site. We will show
you the steps to copy the changing live data back to the production site and to fail back
(see 6.17.9, “How to go back to the production site” on page 366).
6.17.1 Before you start
As described in the PPRC chapter (Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on
page 55), the volume at the primary site which the hosts are writing to is the A volume. This is
copied to the remote site using PPRC-XD and the target is the B volume. The FlashCopies
are taken at the remote site to the C volumes.
For the examples shown in the following section, we have described and set up a single path
to demonstrate the methodology. It is recommended that a minimum of two paths are used for
redundancy. You should consult with your IBM representative to size the amount of bandwidth
you need for your solution (see 4.11, “Asynchronous PPRC Bandwidth Estimator” on
page 119).
For any procedure that is run from the WUI, the last screen that comes up is similar to
Figure 6-176 on page 329, which gives you some options. If the task you are doing is only
being used once or you do not require the ability to re-run it repeatedly, then from this screen
all you do is click the Run button. If the task needs to be repeated, or needs to be run from the
CLI command interface, then you should save it.
Recommendation: It is recommended to use Multiple Device Manager Replication
Manager (see 9.9, “Multiple Device Manager Replication Manager” on page 505) or
Asynchronous PPRC Utilities (see 9.10, “Asynchronous PPRC Utilities for Open System
Environment” on page 507) to manage a large number of volumes in an Asynchronous
PPRC Session. This is due to the fact that during recovery, the state of every volume must
be checked individually to determine its state of consistency and the action to be taken on
each to either revert or commit the FlashCopy. For a small number of volumes, this is
reasonably straight forward using the Web User Interface.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
329
Figure 6-176 Define Task wizard
To save a task, give it a unique task name and add a full text description to help you identify
its function from the task list. Next, click Save to permanently save this task, so it can be run
again at another time. Tasks can also be called from the CLI command interface.
6.17.2 Configuration
For the following demonstrations, the configuration used is a single ESS at the primary site
and a single ESS at the secondary site. One LSS is used at the primary ESS for the A
volumes. At the remote site, a single ESS is also used. The B volumes are in one LSS and
the C volumes are in another LSS of their own (see Figure 6-177 on page 330).
A different configuration will be used only when it is necessary to illustrate the process, such
as when setting up subordinate ESS configurations.
The ESS in the configuration is linked with FCP Host adapters to a common SAN. Only one
PPRC link will be established in the following examples for simplicity. A minimum of two is
recommended in a production environment.

330
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-177 Configuration used for describing Asynchronous PPRC
6.17.3 Setting up Asynchronous PPRC
The steps required to set up a volume or volumes in an Asynchronous PPRC relationship are
as follows:
1.Establish paths from each primary LSS to the secondary LSS that will have a pair.
2.Establish PPRC-XD pairs A

B.
3.Establish FlashCopy pairs B

C.
4.Define an Asynchronous PPRC session.
5.Add the volumes to the session.
Paths
Every PPRC pair relationship needs a PPRC path. These are logical unidirectional
connections between LSSs and can be either over ESCON links or FCP. PPRC paths are
discussed in 6.13.1, “Establishing paths” on page 260. Only FCP paths can be used for
Asynchronous PPRC.
These paths are only in one direction, from the primary volume to the secondary volume. If
PPRC pairs exist in the reverse direction, then a reverse path must be established. Also, the
paths are from one LSS to another, so any volume in an LSS can use a path that exists to the
LSS that its secondary is in, but a new path must exist for every LSS to LSS relationship.
More than one path can be implemented in parallel for each LSS pair for redundancy.
For this example, we will establish one path.
Referring to Figure 6-178 on page 331, click on the Paths button to bring up the Paths
display.
Sample Asynchronous PPRC Configuration
PPRC-XD
active
A
B
C
FlashCopy
SAN

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
331
Figure 6-178 Establishing paths: Paths panel
In the left panel, refer to Figure 6-179, click on the pull-down, and select the source ESS. All
ESSs in this Copy Services Domain will be listed. Now click on the primary LSS that this path
will be from.
Figure 6-179 Establishing Paths: Select primary LSS
Click on the Work with Paths button at the top of the panel, as shown in Figure 6-180 on
page 332.

332
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-180 Establishing Paths: Work with Paths
Next, click on the pull-down in the center panel and select the ESS that contains the
secondary LSS. Click on the secondary LSS (see Figure 6-181).
Figure 6-181 Establishing Paths: Select secondary LSS
The right panel shows the links available. The default display is ESCON. Click on the
pull-down to select FCP, as shown in Figure 6-182 on page 333. Click on the adapter you
wish to use for your link. Now right-click to launch the Task Wizard.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
333
Figure 6-182 Establishing Paths: Selecting path
The task wizard, as shown on Figure 6-183, will launch. Select Establish Paths.
Figure 6-183 Establishing Paths: Establish Paths
If a switch is attached to the port, then the Select outgoing ports wizard will appear, as shown
in Figure 6-184 on page 334. A pull-down of the available ports from the switch will be shown.
Select the correct port for the path you are establishing, and click Next.
Tip: If you want to configure multiple paths at the same time, click on the first path, then
right-click on subsequent paths to enable multiple selection. You need to right-click the last
one highlighted to launch the Task Wizard.

334
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-184 Establishing Paths: Outgoing ports
The Select path options wizard (Figure 6-185) will appear next. This allows for the creation of
a Consistency Group for all PPRC relationships using this path. This is not required for
Asynchronous PPRC, so leave un-selected. Click Next.
Figure 6-185 Establishing Paths: Select path options
The next screen is the Define Task Wizard, as shown in Figure 6-176 on page 329. If you
wish to save this task for later reuse, fill in the fields and click Save. Next click Run.
This will return you to the main panel, as shown in Figure 6-186 on page 335. The path now
has three blue *** underneath it. This indicates that the path is established from the selected
LSS on the left to the selected LSS in the middle.
Tip: To display the a path, the correct primary and secondary LSSs must be selected. The
path panel on the right is unique to the selections made on the left and middle panels.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
335
Figure 6-186 Establishing Paths: Display established paths
Creating PPRC-XD pairs
This portion of Asynchronous PPRC uses the same setup as for a normal PPRC-XD pair.
First, click the Volumes button to display the volumes screen. Click the pull-down on the left
(source) panel and select the ESS:LSS combination that will be your source LSS. Do the
same on the right panel to select the target LSS.
For single selection, click once on the source and right-click once on the target. Now
right-click again on the target to launch the wizard.
You can perform the same operation on multiple volumes. To do this, first click on the Enable
Multiple Selection Mode button at the bottom. Now, click one volume in the left panel, then
right-click on its partner in the right panel. Then click on the next volume source in the left
panel and right-click
its
partner in the right panel. Continue, adding as many pairs as
required. If you do this out of sequence, the panels will reset and all pairs will be lost, so start
again.
Once all the pairs have been highlighted, right-click on the last one clicked on the right panel.
This will launch the wizard, as shown in Figure 6-187 on page 336, where we have
multiselected three volumes.

336
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-187 Creating PPRC-XD pairs: Select volumes
Select Establish PPRC Extended Distance copy pair and click Next.
This will bring up the window shown in Figure 6-188. Select Copy entire volume and click
Next. You will once again get the task save wizard; save this task if you wish. Click Run.
Figure 6-188 Creating PPRC-XD pairs: Copy options
You will now be returned to the Volumes panel and the status of the volumes you selected will
have changed to PPRC-XD, as shown by the icon next to the volume, as in Figure 6-189 on
page 337.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
337
We now have our A volumes, the primary (or source) of the PPRC pairs, and our B volumes,
which are the secondaries (or target). These B volumes are normally located in another ESS
at the remote site.
Figure 6-189 Creating PPRC-XD pairs: View results
Creating FlashCopy pairs
Now that you have the PPRC-XD pairs set up, the next step is to create a FlashCopy
relationship for each of these. This FlashCopy uses the PPRC secondary as its source, and
now a third volume in the remote ESS for the target. The source is our B volume and the
target will be our C volume.
The first step is to select the ESS:LSS on the volume panel for our B volumes, currently the
targets of our PPRC. Next, select the ESS:LSS in the right (target) panel that contains the
volume(s) we will be using or the C volumes. Using either multiple selection, as described
earlier, or individually, click on the source volume on the left and right-click on the target
volume. Once all selections are made, right-click again on the last volume highlighted to
launch the wizard, as shown in Figure 6-190 on page 338.

338
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-190 Creating FlashCopy pairs: Selecting volumes
Select Establish FlashCopy pair from the window and click Next.
This will bring up the copy options task wizard, as shown in Figure 6-191 on page 339. It is
important to select the correct option here. Select the options from this panel as follows:

No background copy: This prevents copying of all the tracks to the target. Only changed
tracks are copied. This is necessary to speed up recovery later.

Persistent FlashCopy: Maintains the relationship after the FlashCopies complete. This
will be forced by the change recording option below anyway.

Inhibit writes to target: Prevents the C volume from being altered. We need this volume
intact as our recovery volume.

Start change recording: This option records changes made to the source after a
FlashCopy has been made, so that when another one is performed, only tracks that
changed in that time interval need to be copied over.
Click Next.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
339
Figure 6-191 Creating FlashCopy pairs: Copy options
The next window allows you to modify the sequence number of the FlashCopies with a prefix.
This can aid in analyzing multiple Consistency Groups in a large enterprise. Leave this at the
default of 0, as shown in Figure 6-192. Click Next.
Figure 6-192 Creating FlashCopy pairs: Sequence number
You will once again get the task save wizard; save this task if you wish. Click Run.
Once you return to the Volumes panel, you will see the combined status, as shown in
Figure 6-193 on page 340. Notice that the B volumes in the left panel show the PPRC-XD
secondary icon, the FlashCopy source icon, and the FlashCopy change recording icon. In the
right panel are our C volumes, showing the FlashCopy target icon and the change recording
icon.

340
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-193 Creating FlashCopy pairs: Viewing results
We now have our volume relationships set up. The next step is to define the session that
Asynchronous PPRC will use to maintain consistency across its members.
Define Asynchronous PPRC session
Asynchronous PPRC uses a session to manage the volumes you wish to remain consistent.
All volumes that are members of the session must have consistent C volumes at the remote
site. This process is controlled by the Asynchronous PPRC Master. One master can exist in
an ESS, although the architecture allows for this to grow. Multiple ESSs can be coordinated
with their volumes in the same session. To do this, the master communicates with a
subordinate function in the other ESSs. Each ESS can only have one Master or Subordinate;
again, the architecture has allowed for growth.
First, we need to open a session for each LSS that contains volumes that will be included in
that session. Then we need to add those volumes to the session. Note that more than one
session can be defined for an LSS, but only one can be active at any time.
To define a session for an LSS, we first click on the Logical Subsystems button to bring up
the LSS window (refer to Figure 6-194 on page 341).

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
341
Figure 6-194 Defining Asynchronous PPRC Session: Selecting LSS
Click on the LSS you are defining to highlight it. Now right-click on it to bring up the wizard, as
shown in Figure 6-195. Select Manage Session and click Next.
Figure 6-195 Defining Asynchronous PPRC Session: Choosing task
The following window gives you several options. You can close a session you already have
defined on this LSS, or you can create one, either with a new session ID, or by using a
session ID already defined in the ESS Copy Services Domain. The pull-down next to the
Select Session Id button will show all sessions currently defined in this domain. This is useful
to make sure that you have all your LSSs in the same session and prevent a typing error. If

342
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
this is a new session, as in our case, then enter a number in the ID entry field at the bottom.
This can be any number from 1 to 255.
When you have the fields filled out, click Next.
Figure 6-196 Defining Asynchronous PPRC session: Select copy options
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the LSS display panel. Repeat this process for every LSS you have
volumes in that are to be included in this session. With a large number of volumes, it will be
very useful to save each of these setup tasks with the save wizard and automate them from
the CLI using scripts.
Next, we need to add our volumes to this newly created session.
Add volumes to session
As discussed in the previous paragraph, we now need to add the volumes individually to the
session we have just created. Again, with a large number of volumes, it will be good practice
to save the tasks as you create them, so they can be automated through the CLI or from the
tasks management.
Click the Volumes button to display the Volumes panel. One by one, or by using the multiple
selection mode, select the volumes you wish to include in this session. Note that with multiple
select, after clicking the Multiple Selection Mode button at the bottom, click on the first
source volume, then right-click subsequent volumes. Right-click on the last volume you
selected to launch the wizard, as shown in Figure 6-197 on page 343.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
343
Figure 6-197 Adding Volumes to session: Selecting volumes
Select Manage Session Members, and click Next.
Figure 6-198 Adding Volumes to session: Selecting the session
From the options window (see Figure 6-198), select Add Volume to Session and from the
pull-down, select a session ID. We have selected the session we created for this LSS above.
You can also use this panel to remove the members from a session. Individual volumes or
groups of volumes can be added or removed from any session, even if the session is running.
Once you have selected the session ID, click Next.
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the Volumes panel, as shown in Figure 6-199 on page 344. Notice that
there is a icon next to the volume indicating that the volume is session join pending (three
small volumes with a larger one beneath and an arrow up). If the session is not yet started, as

344
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
in our case, this will remain in a pending state until the session starts. If the session is already
started, then this icon will not change to the member icon until it is able to form consistent
FlashCopies. This may take a period of time if the PPRC-XD initial copy is still in progress.
Once the volume is able to form a FlashCopy consistent with the other volumes already in the
running session, then the Master will join it to the session. From that time on, it will remain a
member until removed.
Figure 6-199 Adding Volumes to the session: Viewing the status
6.17.4 Starting Asynchronous PPRC
If we have LSSs with sessions of the same ID that we want included in this Asynchronous
PPRC session that are in other ESSs, then we must ensure that there are communication
paths from the ESS with the Master to the ESSs with those LSSs where a Subordinate will be
started. These Master Subordinate communications will pass over any normal PPRC paths
that are setup in the Master ESS to Subordinate ESS direction using any LSS in either ESS.
If a path exists for another PPRC pair relationship, this can be shared; otherwise a new path
needs to be created. Use the create path procedure earlier in this chapter (see 6.13.1,
“Establishing paths” on page 260) or in 6.17.3, “Setting up Asynchronous PPRC” on
page 330 to create these Master to Subordinate paths. They can be from any LSS in the
Master ESS to any LSS in the Subordinate ESS.
Starting a session
Click the Logical Subsystem button to display the LSSs. Choose an LSS in the ESS you
want to be the Master.
Click to highlight it, then right-click to launch the wizard, as shown in Figure 6-200 on
page 345.
Important: Once you select an LSS to be used as communication to the Master, then you
should use the same LSS for all subsequent communications. While it is possible to use
other LSSs in the same ESS, the results may be unpredictable.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
345
Figure 6-200 Starting Asynchronous PPRC session: Selecting the LSS
Select Start/Resume Asynchronous PPRC and click Next. This will bring up the options
wizard, as shown in Figure 6-201 on page 346. Set the timers to your desired values. At the
time of writing, the default options are 0, 50, and 30, respectively (these may change). It is
recommended that you only change the Consistency Group Interval Timer value and leave
the other two as the default unless you have a specific reason to change them. The meaning
of these three timers is as follows:

Consistency Group Interval Timer (in seconds): The time from when the last Consistency
Group was formed (or when the formation was cancelled) to when the next one will be
started. As the Consistency Groups typically form in about 1-3 seconds, then combining
these two values will give the average time between group formations. A value of 0 will
begin forming the next group as soon as the previous one completes.

Maximum Coordination Timer (in milliseconds): The maximum time allowed for the
formation of consistency across all the volumes at the primary site. If this timer times out,
the formation is cancelled.

Maximum Consistency Group Drain Time (in seconds): The time from when the
point-in-time consistency is formed at the primary site, until the Flash Copies have been
started at the remote site. The time to complete the PPRC-XD drain on all volumes.
Select the session ID that this Asynchronous PPRC Master will be managing. Click Next.
Tip: We recommend you use the first LSS in the ESS as a convention to avoid confusion.
Note: Once a session is terminated, it can be restarted on another LSS.

346
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-201 Starting Asynchronous PPRC session: Selecting session and values
The next window allows you to select the ESSs to have Master Subordinate associations with,
as shown in Figure 6-202. These should be ESSs that have LSSs with sessions defined with
the same ID, or will have in the future. Remember that once the session is started, it must be
stopped to change these relationships.
Figure 6-202 Starting Asynchronous PPRC session: Selecting Subordinates
The PPRC paths must already be set up as described above before a subordinate can be
selected. Use the pull downs to create a Master Subordinate relationship for each
Subordinate ESS.
In our example, we are not defining Subordinates, so we simply click Next.
Restriction: Subordinates cannot be added or removed once the session is started. The
session must be terminated and restarted to change the Master to Subordinate
relationships.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
347
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the LSS display, as shown in Figure 6-203. Note that the LSS now has
a different icon. The color and shading still has the same meaning, but the icon has a small
top on it, to indicate it is the Master LSS.
Figure 6-203 Starting Asynchronous PPRC session: Viewing results
We can now display the volumes by clicking on the Volumes button. As can be seen in
Figure 6-204 on page 348, the session icon has changed to session member, as seen by the
double arrow under the three small volumes. If this process was done quickly, there may still
be some delay for each volume to join due to the PPRC-XD pairs still draining. As they are
ready and have formed their first consistent FlashCopy at the remote site, the primary
volumes will show as joined.

348
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-204 Starting Asynchronous PPRC session: Viewing volume status
For the initial start of the session, the second and subsequent Consistency Groups will not be
started until all volumes have completed joining and formed their first consistent FlashCopy.
The status of the Asynchronous PPRC session can be displayed from the LSS view. Click on
the Logical Subsystems button, then click on the Refresh Asynchronous PPRC Data
button at the bottom. A window, as shown in Figure 6-205 on page 349, will be displayed. This
can be left open if desired.
Note: Once the session has all volumes joined and is forming regular consistent copies,
subsequent additions of volumes will not join, but remain join pending, until they are able to
form consistent FlashCopies. They existing volumes in the session will not be interrupted.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
349
Figure 6-205 Starting Asynchronous PPRC session: Viewing status panel
Note the following information:

The Master ESS serial number, the LSS it was started on, and the session ID that it is
using are displayed

The Async Status.

If there has been a fatal error, the reason is displayed; otherwise,
Asynchronous PPRC not
fatal
is displayed.

The settings of the three timers are also displayed.

The time stamp of the last successful Consistency Group and also a percentage of how
many have been successful, followed by the actual number of failures.

How many have been formed since the session was started.

Three error logs, the first error, and the last two are displayed.

Path associations and a display of the number of out-of-sync tracks in the session,
but for
this ESS only
.
6.17.5 Pausing and terminating Asynchronous PPRC
Pausing and terminating a session are done in the same way. Start by displaying the LSS
panel by clicking the Logical Subsystem button. Click to highlight the LSS that the session
was started on. It is easy to identify, as the icon has the yellow top. Refer to Figure 6-200 on
page 345 and Figure 6-203 on page 347 for details.
Select Pause/Terminate Async PPRC from the window, as in Figure 6-206 on page 350.

350
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-206 Pausing session: Task selection
There are three types of task that can be selected on the next window (see Figure 6-207):

Pause: This will stop forming Consistency Groups after the current one is formed.

Terminate: Will interrupt the formation of a Consistency Group if one is running and clean
up by reverting to the previous consistency group. If no group is being formed, it will
terminate immediately. To restart requires issuing a start command and re-establishing the
Subordinate relationships. After a terminate, the LSS no longer shows Master status.

Force Terminate: As for Terminate, but the Asynchronous PPRC session is withdrawn
immediately and will interrupt the creation of the Consistency Group without recovery if
one is being processed.
Figure 6-207 Pausing session: Selecting options and session
In our example, we are performing a pause, so we select Pause in the top section. Select the
session ID and click Next.
At the next screen, refer Figure 6-208 on page 351; the subordinate relationships will be
displayed. Click Next.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
351
Figure 6-208 Pausing session: Selecting Subordinates
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the LSS display. You will note that for a pause the LSS still shows as a
Master, but for a Terminate, the Master icon is no longer displayed.
Figure 6-209 shows the Async PPRC Information Panel after a pause has been completed.
Note that the Async status now shows a state of
Paused
.
Figure 6-209 Pausing session: Viewing status panel
6.17.6 Switching sites after a disaster
There are several forms that a disaster can take and in varying magnitudes. In this section,
we will describe the steps to take at the remote site after the disaster has occurred. It is
assumed that Consistency Groups are no longer being performed and the PPRC relationship
with the primary site is in an unknown state. It may still be running or links might have failed or
suspended. Some of the scenarios that could cause this state are:

FCP links from the primary to the secondary are not operational.

352
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

A freeze has been issued at the primary site, suspending the PPRC relationship.

A pause has been issued, ceasing the formation of consistent copies.

The primary site has had a full disaster and cannot be contacted.

A planned switchover is being performed.
To recover at the remote site, it is necessary to perform a failover to ensure that the PPRC is
suspended and to make the B volumes primary.
Failover
This step must be done for volumes in the session that you wish to recover. It can be
performed at the remote site. Remember that in an LSS, multiple volumes can be selected.
Also if this step is done in testing and the tasks saved, then using these tasks through the CLI
or from the Tasks button can speed up this process and improve accuracy.
Right-click on the last selected volume to launch the wizard, as shown in Figure 6-210. Select
Establish PPRC Extended Distance copy pair and click Next.
Figure 6-210 Failover to secondary: Selecting volumes
On the next screen, as shown in Figure 6-211 on page 353, select the option PPRC Failover.
Click Next.
Important: When performing a failover, the original target becomes the source and vice
versa. So the source is the B volume, with A as the target.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
353
Figure 6-211 Failover to secondary: Selecting options
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the volumes display, as shown in Figure 6-212. Make sure that the
PPRC-XD icon is now blue, indicating it is the primary and the suspended icon is present.
Figure 6-212 Failover to secondary: Viewing results
The B volumes are now accessible from the host systems at the remote site. The state of
these volumes is dependent on why the primary site failed. If it is a planned failure/outage
with knowledge that the data on the B volumes is consistent, then they can be used
immediately, but if this is unplanned or no action was taken to ensure consistency, then the
volumes are an inconsistent state, with no point-in-time consistency.

354
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To recover to the Asynchronous PPRC last consistent copy, the data from the C volumes must
be recovered. To do this, see 6.17.7, “Remote site recovery” on page 354.
6.17.7 Remote site recovery
This section assumes that a failover has been performed and the B volumes are primary and
in a suspended state. The purpose of this section is to recover the consistent data from the C
volumes onto the B volumes for use by the host systems at the remote site.
This will be done in the following steps:
1.Determine the revertible state of the FlashCopy pairs.
2.Revert or commit if required.
3.Fast Reverse Restore the data from the C volumes to the B volumes.
4.Reinstate the FlashCopy pairs to the C volumes.
Determine revertible
Because we cannot determine at what point the Asynchronous PPRC will be in when a
disaster occurs, we must ensure that the process was not in the process of forming
FlashCopies. If the process was between forming Consistency Groups, then the last group
formed will be valid. If it was in the process of forming the FlashCopies, then it is possible that
some copies have been done and not others. All FlashCopies remain in a revertible state until
they all have been completed.
The simplest way to do this is to look at the Information Panel for the FlashCopy pairs and
determine if they are still in a revertible state. If not, which will be true in most situations, then
no action is required. If they are, then the copies must be either Committed to the new and
latest copy or Reverted back to the previous copy.
To display the state, first click on the Volumes button to display the volumes. Click on the
pull-down and select the LSS containing the B volumes, the FlashCopy source. Click on one
volume and then click on the Information Panel button at the bottom right of the window. This
will display a window, as shown in Figure 6-213 on page 355.
In the middle panel, scroll down to display properties of the FlashCopy peer. If the pair is
revertible, then the word
Revertible
will be displayed in the properties area. The volume in
Figure 6-213 on page 355 is not revertible. Also, look at the sequence number above this and
record it. This must be the same for all volumes in the session.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
355
Figure 6-213 FlashCopy status: Not revertible
In Figure 6-214 on page 356, we show an example of a revertible FlashCopy volume. Note
the word
Revertible
in the properties area. Also note the sequence number; this needs to be
recorded, as all volumes must be analyzed and compared with this sequence number. The
number will only be one of two possible sequence numbers: the sequence of the Consistency
Group for the last operation, or that for the current operation.

356
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-214 FlashCopy status: Revertible
If a volume is Revertible, then the sequence number will be that of the new copy. Revertible
means that the FlashCopy process is able to revert back to the volume data of the previous
FlashCopy. It is not normal to see a volume in a revertible state. This only occurs during the
very small instance of time that the FlashCopy operations are being done at the remote site.
Once they are all completed, then they all commit and there is no longer any ability to Revert
or Commit. If a failure occurs during this very small window of time, then the operation may
not be completed and user intervention is required to determine the state and complete the
Commit or Revert.
Volumes can be in one of the following states:

Revertible with the new sequence number. The volume has received the copy instruction
and is waiting for the commit when all volumes are complete.

Non Revertible with the old sequence number. The volume has not received the latest
copy instruction.

Non Revertible with the new sequence number. The copy operation completed and the
commit was received.
The next action is to compare the sequence numbers and revertible status of every volume in
the session. Refer to Table 6-6 on page 357 to decide what action to take. The term equal
refers to the sequence numbers being all the same.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
357
Table 6-6 Revertible status and actions
Once the action has been determined, the individual volumes can be reverted or committed
as described in “Revert or Commit” on page 357. Note that if the volume is not revertible then
no action is required.
Revert or Commit
The commit or revert function is part of the withdraw command. The FlashCopy pair must be
withdrawn and the commit or revert option specified. First, open the volumes display by
clicking on the Volumes button. Select the ESS:LSS that contains the volumes to be
withdrawn. If there are volumes in other LSSs, then this action may have to be repeated
several times.
In the left panel, click on the pull-down and select the ESS:LSS containing the B volumes, the
Flashcopy source. Next, click the pull-down on the right panel and select the C volume LSS.
Click on the left source volume and right-click on its pair in the right panel. You can use
multiple selection mode to select multiple volumes. Right-click on the last volume to launch
the wizard, as shown in Figure 6-215 on page 358. Select Withdraw FlashCopy pair and
click Next.
Indication Specification on FlashCopy Withdraw
Revertible or Non Revertible and Equal Commit
Non Revertible and Equal No Withdraw required
Revertible and Non Revertible Revert
All Revertible Revert
Consistency Group corrupted Manual recovery required
Tip: Not all the volumes have to be committed or reverted the same, as the process
applies to the volume pairs individually. If one or two volumes are on the old sequence
number, but all the volumes for the same host are on the new sequence number, then
those volumes could be committed and the remainder reverted, depending on the data
layout. It is the users responsibility to make this decision to maintain consistency.
Important: Any volume that is in a revertible state
must
be withdrawn with either the revert
or commit option. The volumes remain write inhibited until this is done.

358
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-215 Commit-Revert: Task selection
Depending on your recovery decision, select either Revert the FlashCopy or Commit
FlashCopy, as shown in Figure 6-216.
Figure 6-216 Commit-Revert: Options selection
Next, a window is displayed (refer to Figure 6-217 on page 359), where you enter the
sequence number. If you are using sequence numbers to manage your FlashCopies, enter
the correct number for this volume; otherwise, leave it as the default and click Next.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
359
Figure 6-217 Commit-Revert: Sequence number
You will once again get the task save wizard; save this task if you wish. Click Run.
This completes the procedure. Repeat the above process for every volume in the session that
is in a revertible state. Continue with restoring the C volume data back to the B volumes in the
next section.
Fast Reverse Restore
Once all data has been confirmed to be consistent, the C volumes can be copied back to the
B volumes in readiness for host system use. This process must be performed on every
volume in the session that needs to be recovered. Volumes in the same LSS can be done
together using the multiple selection mode as described earlier.
From the Volumes panel, in the left panel, select the ESS:LSS than contains the B volumes;
this is the FlashCopy current source. In the right panel, select the LSS with the target
volumes.
Click on the new source in the left panel and right-click on the new target in the right panel.
When all are selected, right-click on the last volume in the right window to launch the wizard,
as shown in Figure 6-218 on page 360. Select Establish FlashCopy pair and click Next.
Attention: It is important to select the source and target volumes correctly. The B volume
(original source) must be the source and the C volume must be the target. The process will
complete successfully the other way around, but the data will not be transferred correctly.

360
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-218 Fast Reverse Restore: Selecting volumes
At the next window, shown in Figure 6-219, we have selected Fast Reverse Restore. This
gives the same result as a Reverse Restore, but does not require that the copy to the previous
target completes before copying back. Because we set the FlashCopy up with the options No
background copy, Persistent FlashCopy, Inhibit writes to target, and Start change
recording, then a Fast Reverse Restore is possible.
Note: If the PPRC relationship on the B volume still exists (as in our example) then you will
also need to click the Establish Flashcopy target on existing PPRC Primary or the Fast
Reverse Restore will fail.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
361
Figure 6-219 Fast Reverse Restore: Selecting options
After selecting Fast Reverse Restore, click Next. If you are managing your FlashCopies with
sequence numbers, enter the correct number on the next screen, shown in Figure 6-220 on
page 361; otherwise, leave the default. Click Next.
Figure 6-220 Fast Reverse Restore: Sequence number
This will return you to the volumes display. While the reverse copy is in progress, you will see
the status icons, as shown in Figure 6-221.

362
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-221 Fast Reverse Restore: Viewing progress
When the copies are complete, the status icons will be removed and the display will show no
FlashCopy relationship, as in Figure 6-222 on page 362.
Figure 6-222 Fast Reverse Restore: Viewing results
Important: The data on the old FlashCopy target, the C volume, is indeterminate and no
longer valid after a Fast Reverse Restore. The data must be rewritten before the volumes
are used.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
363
Next, we will re-instate the original FlashCopy pairs. This leaves the volumes ready for the
resumption of Asynchronous PPRC in the future, puts valid data back on the C volumes, and
also recreates the consistent copies in case they are needed again. This might be necessary
if the application or system recovery fails. It will then be possible to perform the Fast Reverse
Restore again and redo the system recovery. This process, therefore, can be done as many
times as necessary to achieve a successful recovery of the systems.
Establish the FlashCopies from the B volumes to the C volumes as they were originally setup,
using the same parameters and options. Use the procedure we described earlier in “Creating
FlashCopy pairs” on page 337.
Once complete, you should have the display shown in Figure 6-223 on page 363.
Figure 6-223 Fast Reverse Restore: Re-instating FlashCopy pairs
6.17.8 Planned copy at remote site
If there is a requirement to take a point-in-time copy of the primary site at the secondary site,
then Asynchronous PPRC is ideal, particularly if you already have it set up for Disaster
Recovery. Asynchronous PPRC is taking consistent point-in-time copies at the remote site at
regular intervals, as determined by the Consistency Groups Interval Timer.
It is therefore quite simple to freeze one of these sets of copies and make local copies at the
remote site. This might then be used for backups, testing, quality testing, disaster testing, and
so forth.
To do this, we issue a pause command to the Asynchronous PPRC Master. This causes it to
pause after the formation of the current Consistency Group, so it will stop forming them. The
last one formed is then available at the remote site.
Attention: Perform the FlashCopy back to the C volumes before any data is changed on
the B volumes to ensure that the original consistent data is retained.

364
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Because we probably do not want to leave the primary systems without this copy process
running for any length of time, we need to make separate copies of the data at the remote site
by using FlashCopy to another set of volumes, then restart the Asynchronous PPRC session.
The steps will be:
1.Pause the session.
2.Perform recovery at the remote site as for a disaster.
a.Failover to the B volumes.
b.Confirm sequence numbers (should not be a problem after a pause).
c.Fast Reverse Restore.
d.Re-instate the B to C FlashCopies.
3.FlashCopy the B volumes to another set of volumes (D volumes).
4.Failback A to B, to restart the PPRC-XD pairs.
5.Resume the session.
Alternate methods (though more disruptive) of forming consistent data at the remote site are
as follows:

Change the PPRC-XD pairs to synchronous mode, and perform a freeze at the primary
site. This will stop I/Os at a point-in-time, and the data on both the primary and secondary
volumes will be consistent. The PPRC relationships for all volumes will be suspended,
leaving a consistent set of data at the secondary site. This can then be FlashCopied to the
D volumes and the Asynchronous PPRC environment restarted.
This process would cause considerable I/O response degradation at the primary site,
assuming that there is a reasonable distance between the sites.

If the applications allowed for an outage, then I/O could be temporarily quiesced. Then the
PPRC-XD pairs could go to synchronous and a freeze issued. After this, the I/O could be
restarted.
Pause Asynchronous session
The first step is to pause the running session and stop forming Consistency Groups, leaving a
good one on the C volumes.
This step is described in 6.17.5, “Pausing and terminating Asynchronous PPRC” on
page 349. Perform this step and return here.
Recover volumes at the remote site
This step will first fail over to B, then confirm the status of the C volumes with regard to
consistency. After that, the C volumes are copied back on to the B volumes, and then the C
volumes are restarted as FlashCopy pairs.
We have already described this process in detail. First, perform the fail over, as described in
6.17.6, “Switching sites after a disaster” on page 351, and then perform the recovery steps,
as described in 6.17.7, “Remote site recovery” on page 354.
Make working copies, D volumes
This involves making a working set of volumes available and performing a FlashCopy to them.
We will use the copy option so they can be isolated from the Asynchronous PPRC
relationship.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
365
Click the Volumes button to display the volumes. In the left panel, select the LSS for the B
volumes and in the right panel, select the LSS that contains the empty volumes that will be
used for the D volumes. Note this must be in the same ESS. Using multiple selection mode or
individually, click on the B volume in the right panel and right-click on the new D volume in the
right panel. Right-click again on the last volume to launch the wizard.
From the copy options window, shown in Figure 6-224 on page 365, do not select any
options; leave as the defaults. This will perform a single copy only, which will copy all data to
the target, and then complete.
Figure 6-224 FlashCopy to D: Select volumes
At the next window (Figure 6-225), leave the default and click Next.
Figure 6-225 FlashCopy to D: Select options
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the Volumes display. The status icons will show that the FlashCopy has
been done and the copy of the data to the D volumes is in progress. Note also that because
we still have the FlashCopy to the C volumes, that the multiple FlashCopy icon is displayed for
the source volumes.

366
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 6-226 FlashCopy to D: View results
When the copy of the data to the D volumes has completed, these icons will disappear, and
the status for the B volumes will be as before.
6.17.9 How to go back to the production site
Once the systems have been brought up at the remote or recovery site, the data is being
changed and the original data at the production site is no longer current. Depending on the
length of the disaster, it may be some time before the systems are returned to their normal
place. The modified data at the remote site now needs to be copied back to the primary site
(the reverse of the function just performed). We will use the same PPRC relationships used
for the failover to return the data. This will also ensure that the volumes are returned to their
original locations.
The steps required to perform these tasks are:

Paths: The PPRC paths must be created in the reverse direction.

Failback: Using the original PPRC relationships, the data is copied back to the A volumes.

Quiesce: When ready, the recovery systems are quiesced or shut down.

Go-to-sync: Ensures that all data is at the local site and is consistent.

Failover: Make the A volumes primary and ready for use.

Reestablish: Re-instate the Asynchronous PPRC environment.
Paths
To be able to establish the PPRC relationship in the reverse direction back to the local or
primary site, we must first ensure we have PPRC paths in that direction. This is done in the
same way as before, but the source is the LSS at the remote site and the target is the LSS at
the local site. Paths must be established for every LSS to LSS relationship.
Open the paths display by clicking on the Paths button. In the left panel, using the pull-down,
select the ESS and click on the LSS that the B volumes are in. Click the Work with Paths

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
367
button at the top. In the middle panel, select the ESS for the A volumes and click on the LSS
to highlight. In the right panel, select FCP from the pull-down to display the available
adapters. To create a path, click on the adapter, then right-click to launch the wizard, as
shown in Figure 6-227.
Figure 6-227 Failback to primary: Paths, selecting LSSs
Select Establish paths and click Next.
From the next window (Figure 6-228), select the destination port and click Next.
Figure 6-228 Failback to primary: Paths, selecting ports
You will once again get the task save wizard; save this task if you wish. Click Run.

368
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
On the paths display window, confirm that the path has established by looking for the three
blue *** directly under the path in the right panel.
Failback
The following step assumes that the PPRC relationship still exists between all volumes and
that the PPRC peers are in a suspended state. If any PPRC relationships have been lost,
then the PPRC relationship needs to be re-established. For this step, this can be done by
establishing the PPRC-XD pairs from the B volumes to the A volumes in the normal manner.
For any volumes that are done this way, skip this step.
Go to the volumes display and click on the Volumes button. In the left panel, select the
ESS:LSS that contains the B volumes. In the right panel, select the ESS:LSS for the A
volume. Individually or using multiple selection mode, click on the B volumes in the left panel
and right-click on the A volumes in the right panel. Right-click on the last volume again to
launch the wizard, as shown in Figure 6-229.
Select Establish PPRC Extended Distance copy pair and click Next.
Figure 6-229 Failback to primary: Selecting volumes
From the options panel shown in Figure 6-230 on page 369, select PPRC Failback. This
option will establish a PPRC-XD relationship from the B to the A volumes. The B volumes will
be primary and the A secondary. The data from the B volumes will be copied to the A
volumes. If the volumes were still suspended, then only changed data will be actually copied,
speeding up the process. Any data changed on the A volumes, since the last Consistency
Group was formed, will be lost.
Click Next.
Important: Confirm that the PPRC relationships still exist for all volumes in the session. If
not, then establish PPRC-XD peers from B to A for those volumes.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
369
Figure 6-230 Failback to primary: Selecting options
You will be returned to the Volumes display and the status icons will show a normal PPRC-XD
relationship, as shown in Figure 6-231.
Figure 6-231 Failback to primary: Viewing results
Quiesce
View the status on the background copies of the volumes by displaying the Information panel
with a volume selected. In the PPRC Peer section, look at the number of out-of-sync tracks or
sectors. This will reduce as the volumes drain and catch up. When this has stabilized for all
volumes, you are ready to go back to the primary site.

370
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To ensure that all data is copied over when we transfer, we must ensure that the PPRC
volumes are equal. There are two ways to do this:

Stop the I/O at the remote site and manually check to ensure all the volumes have no
out-of-sync tracks or sectors.

Change the PPRC relationship to synchronous mode. This may have an effect on the
response times for the hosts at the remote site.
Unless the I/O requirements of the system can be reduced to a very low level and there is a
reasonable distance between the sites, the going to synchronous mode on a running system
is generally not acceptable.
Therefore, it is necessary to plan the changeover with a period of no activity before the
change occurs. This will involve quiescing or shutting down the systems. Once this is done, it
needs to be confirmed that the volumes are equal before changing over. Checking the
out-of-sync status will be time consuming and subject to error. The most reliable method is to
change all the PPRC relationships to synchronous mode, then, when they are all
synchronized, the change can be performed with confidence.
For our example, we will quiesce the systems and then go to synchronous mode. Systems
can be shut down or quiesced to allow a quick restart in case the changeover process is
canceled.
Go to synchronous mode
From the Volume display, use the pull-down to display the PPRC source volumes, B, in the left
panel and the corresponding target volumes, A in the right panel. Using multiple selection
mode or individually, click on the source volumes in the left panel and right-click on the target
volumes in the right panel. Right-click again on the last volume to launch the wizard.
Select Establish Synchronous PPRC copy pair and click Next, as shown in Figure 6-232.
Figure 6-232 Failback to primary: Go to synchronous, selecting volumes

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
371
The transition from PPRC-XD to Synchronous PPRC can be done without suspending or
withdrawing the relationship. It is only necessary to copy the out-of-sync-cylinders, as shown
in Figure 6-233.
Figure 6-233 Failback to primary: Go to synchronous, selecting options
Click Next.
You will once again get the task save wizard; save this task if you wish. Click Run.
On the Volumes display, you will see the status icons change to the synchronous draining
icon, which is the single volume symbol half full. When all the tracks have been copied and
the updates are running in full synchronous mode, the icons will change to solid, as shown in
Figure 6-234.
Figure 6-234 Failback to primary: Go to synchronous, viewing results

372
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
This process needs to be repeated for all volumes in the session. When all volumes are in
synchronous state and the systems are quiesced, you are ready to perform the failover.
Failover
We are now ready to failover the volumes back to the primary site. This action will suspend
the PPRC relationship and make the A volumes into primary volumes and accessible to the
hosts.
Click on the Volumes button to show the volumes display. In the left panel, select the local
LSS that contains the A volumes and, in the right panel, select the corresponding remote
panel that contains the B volume peers. Either using multiple selection mode or individually,
click on the source, A, volume in the left panel and right-click on the target, B, volume in the
right panel. Right-click again on the last volume to launch the wizard, as shown in
Figure 6-235.
Figure 6-235 Failback to primary: Failover, selecting volumes
Click Establish Synchronous PPRC copy pair and click Next.
From the options window, select PPRC Failover and click Next, as shown in Figure 6-236 on
page 373.
Tip: It is advisable to prove that the systems are able to recover and that the changeover is
committed to the primary site before establishing the PPRC pairs back in the normal
direction, which will overwrite the B volumes. This allows for the option to cancel the
change and return to the remote site if required.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
373
Figure 6-236 Failback to primary: Failover, selecting options
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the volumes display where you will see that the A volumes are now
primary suspended, but the B volumes have not changed state, as shown in Figure 6-237.
Figure 6-237 Failback to primary: Failover, viewing results
Host systems can access the A volumes. Bring up the host systems and test before taking the
next step, which will overwrite the B volumes at the remote site.
Reestablish Asynchronous PPRC
When all systems are recovered, Asynchronous PPRC can be configured and restarted. The
PPRC relationships are still in place but suspended. The FlashCopy relationships were
re-instated as part of the Fast Reverse Restore and will be correct. If there are any problems
with any of these, then set up the Asynchronous PPRC from the start, as described in 6.17.3,
“Setting up Asynchronous PPRC” on page 330.

374
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To reestablish the PPRC-XD peers back to the B volumes, we do not need to start from the
beginning, as the volumes are still in a relationship, but suspended. The volumes will be doing
change recording, so a Failback command is used to resynchronize them. We will be
performing a Failback from the A volumes, as the source, to the B volumes, as the target.
Click on the Volumes button to show the Volumes display. On the left panel, select from the
pull-down the LSS that contains the A volumes at the primary site. On the right panel, select
the remote LSS for the B volumes.
Using multiple selection mode if you wish, click on the A volume in the left panel and
right-click on the B volume in the right panel. Right-click again on the last volume to launch
the wizard, as shown in Figure 6-238.
Figure 6-238 Reinstating PPRC: Selecting volumes
Select Establish PPRC Extended Distance copy pair and click Next.
From the next window, select PPRC Failback and click Next, as shown in Figure 6-239 on
page 375.
Important: Make sure that you choose PPRC-XD. You will have to suspend the pairs to
change this if you select synchronous.

Chapter 6. ESS Copy Services Web User Interface: LIC 2.2.0 and later
375
Figure 6-239 Reinstating PPRC: Selecting options
You will once again get the task save wizard; save this task if you wish. Click Run.
This will return you to the Volumes display, as shown in Figure 6-240. This shows the PPRC
pairs are in PPRC-XD mode and the FlashCopy pairs are also correctly configured.
Figure 6-240 Reinstating PPRC: Viewing results
The next step is to start or resume the Asynchronous PPRC session. If it was paused as part
of the disaster, then it can be resumed at this point using the procedure described in 6.17.4,
“Starting Asynchronous PPRC” on page 344. If it has been terminated, the sessions may
need to be defined again and the volumes added. Then the session can be started as
described above.

376
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
6.18 Summary of the benefits of the WUI
Starting with LIC Version 2.2.0, the WUI provides an enhanced Copy Services Web User
Interface, compared with the WUI provided with LIC versions prior to 2.2.0.
The main differences are:

New Tools panel with new options
– The user can define the active servers of the Copy Services Domain (“Defining the
active servers” on page 193).
– The user can define the clients in the Copy Services Domain (“Defining the clients” on
page 194).
– The user can perform a Domain-wide Reset (“Domain-wide Reset” on page 196).

New way to start the Copy Services WUI
Since LIC 2.2.0 or higher provides a new dual-active mode, you can now choose from
which server you want to start the WUI (6.4, “Connecting to the Copy Services Web User
Interface” on page 196).

New Volume panels in the WUI
New volume presentations with new icons. The volumes are now presented in rows. When
the volume is involved in one or more Copy Services relationships, multiple icons will
identify each relationship and the state of the volume (see 6.5, “Volumes panel” on
page 199).

New option for PPRC pairs
To benefit from the new option, you must have the PPRC V2 feature installed. The new
option is Asynchronous Cascading PPRC (see 6.16, “Establishing an Asynchronous
Cascading PPRC” on page 300).

New option for FlashCopy pairs
To benefit from the new option, you must have the FlashCopy V2 feature installed.
The new options are:
– Source and target volumes can be in a different LSS in the same ESS.
– Multiple FlashCopy using the same Source volume (6.11.4, “Multiple FlashCopy using
a single source volume” on page 237).
– Incremental FlashCopy (6.11.6, “Incremental FlashCopy” on page 240).
– Reversing a FlashCopy (6.11.7, “Reversing a FlashCopy relationship” on page 246).
– Creating FlashCopy Consistency Group (6.11.8, “Creating a FlashCopy Consistency
Group” on page 251).
– Inband Commands (6.11.11, “Inband FlashCopy” on page 256).
With the availability of LIC level 2.3.0, all the previous benefits apply, plus the ability to
configure PPRC using Fibre Channel links and also having double the number of ESSs in a
Copy Services Domain.
With LIC 2.4.0, the panels and options were updated for Asynchronous PPRC.

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
377
Chapter 7.
ESS Command Line Interface
The IBM Enterprise Storage Server (ESS) provides two Command Line Interfaces (CLIs) for
different host platforms:

ESS Copy Services CLI: The ESS Copy Services CLI allows you to communicate with
the ESS Copy Services server from the host’s command line. An example would be
automating FlashCopy by invoking the Copy Services tasks using CLI commands with
customized scripts.

ESS Storage Management CLI: The ESS Storage Management CLI allows you to create
automated routines to manage the storage subsystem and to change the logical
configuration of the ESS.
In this chapter, we describe how to use the ESS Copy Services CLI and we describe the most
useful commands regarding the management of the ESS Copy Services. We also present the
ESS Storage Management CLI.
7

378
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
7.1 ESS Copy Services CLI
The Copy Services Command Line Interface (CLI) is an interface that allows you to
communicate with the ESS Copy Services server from the host’s command line.
You can use the Copy Services CLI to perform several tasks:

Execute one or more ESS Copy Services tasks that you have defined using the ESS Copy
Services Web User Interface (WUI).

Display the mapping of a host disk name to a 2105 volume serial number.

Create, update, or remove a list view of volumes associated with a particular host in the
ESS Copy Services server; you can see the same view in the Volumes panel of the ESS
Copy Services Web User Interface (WUI).

Query the status of one or more volumes.

Access the ESS Copy Services server tasks, as defined and saved within the ESS Copy
Services Web User Interface, and determine whether the volumes defined in a given task
have completed the initial PPRC synchronization.

Query a volume or group of volumes to retrieve FlashCopy data.

Determine whether you can successfully connect to the ESS Copy Services server.
The ESS CLI is available for selected open servers:

IBM AIX

HPUX

Linux (Red Hat, RHEL, SUSE LINUX, SLES)

Sun Solaris

Windows NT 4.0, Windows 2000, Windows Datacenter, and Windows 2003

OpenVMS Alpha

HP Tru64

IBM NumaQ

Novell Netware
For the latest information about supported operating systems versions, refer to the IBM ESS
interoperability matrix, which can be found at:
http://www.storage.ibm.com/disk/ess/ess750/supserver.htm
http://www.storage.ibm.com/disk/ess/ess800/supserver.htm
Detailed and complete information on the ESS Copy Services Command Line Interface is
provided in the IBM TotalStorage Enterprise Storage Server Command-Line Interface User’s
Guide, SC26-7494.
7.1.1 Requirements for Copy Services CLI
The Copy Services Command Line Interface is Java based, and therefore the Java Runtime
Environment needs to be installed on each host system from which you want to issue the
commands.
Note: The CLI version discussed is the latest version available at the time of the writing of
this redbook. The CLI version is bound to the LIC level, so if your ESS has an older LIC
level installed, it might be possible that some CLI commands are different or maybe do not
exist.

Chapter 7. ESS Command Line Interface
379
The use of the Command Line Interface requires, at a minimum, Java JRE level 1.1.8. It is
also supported with Java level 1.3. For current recommendations, refer to the IBM
TotalStorage Enterprise Storage Server Command-Line Interfaces User’s Guide,
SC26-7494.
On AIX, Windows, SUN Solaris, Linux (Red Hat or SUSE), and OpenVMS, the CLI installation
process automatically installs a JRE on your host system. If you do not accept this option, you
must install the required Java on your host system.
To be able to use the CLI, the host system does not necessarily need to be connected to the
storage subsystem using host ports of the ESS. The only requirement is that the server from
which you want to invoke the CLI commands is connected to the ESS that is defined as the
Primary Copy Services server via a local area network (LAN). However, commands that
relate the ESS volume serial number to the host’s physical volume name or number (for
example, the hdisk number in the case of AIX), such as the
rsPrimeserver
and
rsList2105s

commands, will only work on a host system that is physically connected to the ESS storage.
Because the CLI commands need to communicate with the Copy Services server, you have
to identify the Copy Services server before using any of the commands. For instructions on
how to do this, refer to 5.3.2, “Location of ESS Copy Services server” on page 128.
Optionally, it is possible to authorize the usage of the Copy Services commands by specifying
a user and its password when invoking the commands. The administration of these users is
done from the Administration panel of the Copy Services Web User Interface (see 5.8.3,
“Managing the CLI user ID and password for an open-systems host” on page 149). In this
case, the user authorization for the Copy Services CLI needs to be enabled, so you have to
create new users for that purpose. The accounts already created for the ESS Specialist to
administrate the ESS and its storage cannot be used for the Copy Services CLI commands.
7.1.2 Installing Copy Services CLI
The Command Line Interface program is located on one of the CDs, which are delivered with
your ESS or with a new LIC version. To install the CLI, you must be logged on to the
respective platform with the highest administrative privilege (root, administrator, or
supervisor).
For specific operating system installation information, refer to the IBM TotalStorage Enterprise
Storage Server Command-Line Interfaces User’s Guide, SC26-7494.
7.1.3 General command guidelines
In general, all flags can be specified in any order on the command line. The specification of
duplicate command flags is not allowed.
The Command Line Interface is case sensitive. For example, if you specify a task name for
rsExecuteTask
, the case is significant. Use the same upper/lower case combination as
defined in the Web User Interface. Also, on UNIX systems, the file names are case sensitive.
Place the user name, password, task name, and host name parameters inside quotation
marks when specified on an OpenVMS host.
If an argument value contains any embedded blanks, the value must be surrounded by
quotes, either single or double quotes, depending on the operating system being used, for
example:
-a "c:\data files\esscli\ess access file.dat"

380
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
7.1.4 Copy Services CLI Command description
The Copy Services CLI provides seven commands you can use to monitor and manage
predefined ESS Copy Services tasks. In the following section, we describe these commands
and show their name variations.
Table 7-1 provides an overview of the platform specific notations.
The name of the Copy Services CLI commands are the same for all the supported platforms.
Only the CLI command for Novell 4.x differs slightly. For example, use
rsExTask
instead of
rsExecuteTask
on a Novell 4.x system.
Table 7-1 Platform specific notations
rsExecuteTask (rsExTask) command
The
rsExecuteTask
command executes one or more ESS Copy Services tasks, as defined
and saved using the ESS Copy Services Web User Interface, and it waits for the given task to
complete execution.
Successful completion of the task means that the copy pair relationship has been established,
suspended, or terminated (it does not, for example, mean that the background copy task for
FlashCopy has finished).
Use the
rsQueryComplete
command to determine if the PPRC or FlashCopy pairs have
completed the physical copy process.
Syntax of rsExecuteTask command
The syntax for the
rsExecuteTask
command is:
rsExecuteTask.* [-v] {-u username -p password |-a securityfilepath} -s primaryserver
[-b backupserver] tasknames
In this command, the asterisk (*) represents the file extension used on the particular
operating system. For further information, refer to Table 7-1.
Commands File ext.Flag Host systems Example
rsExecuteTask
rsFlashCopyQuery
rsList2105s
rsPrimeServer
rsQuery
rsQueryComplete
rsTestConnection
.sh - AIX, Tru64, Sun,
NUMA-Q®, Linux,
and HP-UX
rsExecuteTask.sh -a
securityfilepath -s
primaryserver
- OpenVMS and
Alpha
rsExecuteTask -u "username
"-p "password "
.exe/Win NT, Win 2000,
Win 2003, and
Datacenter
rsExecuteTask.exe /a
securityfilepath /s
primaryserver
.nlm/Novell 5.x and
Novell 6.x
rsExecuteTask.nlm /a
securityfilepath /s
primaryserver
rsExTask
rsL2105s
rsPSrv
rsQ
rsQComp
rsTConn
.nlm/Novell 4.x
rsExTask.nlm /a
securityfilepath /s
primaryserver

Chapter 7. ESS Command Line Interface
381
Flags and parameters for rsExecuteTask command
You can specify one or more of the following flags and parameters when you issue the
rsExecuteTask
command:
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
taskNames Specifies the name of one or more previously saved ESS Copy
Services tasks. Separate the task names with a space when you
specify more than one task.
Example
Example 7-1 on page 382 shows the output you might see when you invoke the
rsExecuteTask
command on an AIX host system.

382
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Example 7-1 Output of rsExecuteTask command
# ./rsExecuteTask.sh -v -u yourusername -p yourpassword -s primaryservername es_pair12
rsExecuteTask: Got task manager reference
rsExecuteTask: *************Finding the tasks****************
rsExecuteTask: Task es_pair_12 found by task manager
rsExecuteTask: *************Scheduling the tasks****************
rsExecuteTask: Task es_pair_12 scheduled with copy services server
rsExecuteTask: *************Monitoring the tasks****************
rsExecuteTask: Waiting on server...
rsExecuteTask: Task es_pair_12 completed successfully
rsExecuteTask: Command successful
rsFlashCopyQuery command
The
rsFlashCopyQuery
command synchronously queries the user-specified volume or group
of volumes to retrieve FlashCopy data. This data includes FlashCopy peers, out-of-sync
tracks, sequence numbers, date established, date synced, and attributes.
This command is also very useful in the Asynchronous PPRC environment (which comes with
LIC level 2.4.x or higher and PPRC V2). You can query the volumes that are in a FlashCopy
relationship for information like FlashCopy sequence number or if the FlashCopy is
“revertible” (for more information about Asynchronous PPRC, see 4.5, “Asynchronous PPRC”
on page 74).
Syntax of rsFlashCopyQuery command
The syntax for the
rsFlashCopyQuery
command is:
rsFlashCopyQuery.* [-v] {-u username -p password |-a securityfilepath} {-f filename |-q
volumename} -s primaryserver [-b backupserver]
In this command, the asterisk (*) represents the file extension used on the particular
operating system. For further information, refer to Table 7-1 on page 380.
Flags and parameters
You can specify one or more of the following flags and parameters when you issue the
rsFlashCopyQuery
command:
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for CLI commands and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for the ESS Copy Services
server has enabled password protection for CLI commands and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password that are defined at the ESS Copy Services server for using
Copy Services CLI commands on a host system. The a flag with the

Chapter 7. ESS Command Line Interface
383
securityfilepath parameter is required if the administrator for the ESS
Copy Services server has enabled password protection for CLI
commands and if the u and p flags and associated parameters are not
used.
The format of the record in the security path file is as follows:
username password
f filename
Specifies the name of a file that contains vpath names, host volume
names, and volume serial numbers, all of which are to be queried. Use
the f flag with the filename parameter if the q flag with the volume
parameter is not used.
The format of the filename file can be either:
sourceVolume1 targetVolume1
sourceVolume2 targetVolume2
.......................
sourceVolumeN targetVolumeN
or
volume1
volume2
volume3
...........
volumeN
q volumename
Specifies the vpath name, host volume name, or volume serial
number. Use the q flag with the volume parameter if the f flag with the
filename parameter is not used.
s primaryserver
Specifies the IP address or the complete host name of a ESS Copy
Services server. Before you use the s flag with primaryserver
parameter, you must identify and configure an ESS Copy Services
server.
b backupserver
Specify the IP address or the complete host name of an alternate ESS
Copy Services backup server. The b flag with the backupserver
parameter is optional.
Example
Example 7-2 shows the output you might see when you invoke the
rsFlashCopyQuery

command on an AIX machine.
Example 7-2 Output of rsFlashCopyQuery command
#./rsFlashCopyQuery.sh -u username -p password -q 00224663 -s servername
-----------------
24663:10:02
-----------------
Volume Serial = 00224663
State = Source

384
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Peer = 00324663
Out of Sync Tracks = 0
Sequence Number = 46
Date Established = Thu May 06 14:39:19 MST 2004
Date Synced = Thu May 06 14:39:47 MST 2004
Attributes = INHIBIT_WRITES_TARGET BACKGROUND_COPY VOLUME_FLC_PERSISTENT
CHANGE_REC_ENABLED REVERTIBLE
The sample output contains the following fields and values:

Volume Serial: Specifies the serial number of the volume.

State: Specifies one of the following states:
– Source
– Target
– None

Peer: Specifies the peer of the FlashCopy volume that is being queried.

Out of Sync Tracks: Specifies the number of tracks that have yet to be copied.

Sequence Number: Show the sequence number of the FlashCopy pair. This is mainly
useful in the Asynchronous PPRC environment.

Date Established: Specifies the date when the FlashCopy was established.

Date Synced: Specifies the date when the last background copy was completed.

Attributes: Specifies the attributes of the FlashCopy pair.
rsList2105s (rsL2105s) command
The
rsList2105s
command displays the mapping of a host disk name to a 2105 volume serial
number.
If you use the
rsList2105s
command on an OpenVMS Alpha host system, which is a member
of an OpenVMS cluster, the output does not display information about the following devices:

ESS volumes to which the host system has only MSCP paths

ESS volumes to which the host system uses only MSCP paths at this time, even though it
has both MSCP and direct paths
Syntax of rsList2105s command
The syntax for the
rsList2105s
command is simply:
rsList2105s.*
In this command, the asterisk (*) represents the file extension used on the particular
operating system. For further information, refer to Table 7-1 on page 380.
Flags and parameters for the rsList2105s command
The
rsList2105s
command does not have any flags or parameters.
Example
If the Subsystem Device Driver (SDD) is installed, you might see output similar to
Example 7-3 on page 385 when you invoke the
rsList2105s
command on an AIX host
system.

Chapter 7. ESS Command Line Interface
385
Example 7-3 Output of rsList2105s command (SDD installed)
#./rsList2105s.sh
VpathName Serial VolumeNames
----------- ------ -----------
vpath0 00224663 hdisk3 hdisk6 hdisk9 hdisk12
vpath1 00324663 hdisk4 hdisk7 hdisk10 hdisk13
vpath2 00424663 hdisk5 hdisk8 hdisk11 hdisk14
If SDD is not installed, you might see output similar to Example 7-4 when you invoke the
rsList2105s
command on an AIX host system.
Example 7-4 Output of rsList2105s command (SDD not installed)
#./rsList2105s.sh
disk name 2105 serial number
--------- ------------------
hdisk14 40EFC102
hdisk15 40FFC102
hdisk17 40CFC102
hdisk18 40DFC102
rsPrimeServer (rsPSrv) command
The
rsPrimeServer
command creates, updates, or removes a list view of volumes associated
with a particular host in the ESS Copy Services server. You can see the same view in the
Volumes panel of the ESS Copy Services Web User Interface.
This command is useful when you use the ESS Copy Services Volumes panel to perform
FlashCopy, PPRC, or both, because it shows you all the volumes that are connected to the
selected host in one view.
If SDD is installed on your host system and you change the configuration for the host (such as
adding or removing volumes), you must perform the following steps:
1.Use a platform specific procedure so SDD can recognize the addition or removal of the
paths.
2.Run the
rsPrimeServer
command to generate the current mapping for the host volumes.
If SDD is not installed on your host system and you change the configuration for the host
(such as adding or removing volumes), you must run the
rsPrimeServer
command to
generate the current mapping for the host volumes.
If you use the
rsPrimeServer
command on an OpenVMS Alpha host system with MSCP-only
access to some ESS volumes, these volumes are not shown in the host-specific volume list
view in the ESS Copy Services server.
Syntax of rsPrimeServer command
The syntax for the
rsPrimeServer
command is:
rsPrimeServer.* [-v] {-u username -p password |-a securityfilepath} -d hostname -s
primaryserver [-b backupserver]
In this command, the * represents the file extension used on the particular operating system.
For further information, refer to Table 7-1 on page 380.

386
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Flags and parameters for rsPrimeServer command
You can specify one or more of the following flags and parameters when you issue the
rsPrimeServer
command:
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password-protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
d hostname
Specifies the IP address or host name to be removed. The host name
must exactly match one of the listing entries on the Volumes panel of
the ESS Copy Services Web User Interface. The d flag with the
hostname parameter is required when you are removing the host
name.
Example
Example 7-5 on page 387 shows the output that you see when you invoke the
rsPrimeServer

command to add the AIX machine host name to the Volumes panel of the Web User
Interface.

Chapter 7. ESS Command Line Interface
387
Example 7-5 Output of rsPrimeServer command when used to add a host name
# ./rsPrimeServer.sh -v -u yourusername -p yourpassword -s primaryservername
rsPrimeServer: Local host is abc.def.ghi.com
rsPrimeServer: Got task manager reference
rsPrimeServer: Host abc.def.ghi.com registered
rsPrimeServer: Configuration change requested made to primaryserver
rsPrimeServer: Command successful
Example 7-6 shows the output that you might see when you invoke the
rsPrimeServer

command to delete a host name on an AIX host system.
Example 7-6 Output of rsPrimeServer command when used to delete a host name
# ./rsPrimeServer.sh -v -u yourusername -p yourpassword -d abc.def.ghi.com
-s primaryservername
rsPrimeServer: host to unregister is abc.def.ghi.com
rsPrimeServer: Got task manager reference
rsPrimeServer: Host abc.def.ghi.com unregistered
rsPrimeServer: Configuration change requested made to primaryserver
rsPrimeServer: Command successful
rsQuery (rsQ) command
The
rsQuery
command queries the status of one or more volumes. You can invoke the
rsQuery
command for the vpath name, the host volume name, or the volume serial number
under various conditions.
If the SDD is installed, use the
rsQuery
command for the vpath name or volume serial
number. If the SDD is not installed, use the
rsQuery
command for the host volume name or
volume serial number.
The
rsQuery
command fails on an OpenVMS host system if you specify the -m flag for a
volume to which the host system has MSCP-only access. In other words, if your OpenVMS
host system only has MSCP access to a volume, specify the volume serial number when
using the
rsQuery
command.
Syntax of rsQuery command
The syntax for the
rsQuery
command is:
rsQuery.* [-v]-m {-u username -p password | -a securityfilepath} {-q volumename | -f
filename} -s primaryserver [-b backupserver]
In this command, the asterisk (*) represents the file extension used on the particular
operating system. For further information, refer to Table 7-1 on page 380.
Flags and parameters for rsQuery command
You can specify one or more of the following flags and parameters when you issue the
rsQuery
command:
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter
m Maps host volume names to 2105 volume serial numbers. This flag
allows host volume names to be used as a volume flag. Use the m flag
if you use the
rsQuery
command for the vpath name of host volume
name. Do not use the m flag if you use the
rsQuery
command for a
serial number.

388
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password-protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
q volumename
Specifies the vpath name, host volume name, or volume serial
number. Use the q flag with the volume parameter if the flag with the
filename parameter is not used.
Use the m flag if you use the
rsQuery
command for the vpath name or
the host volume name. Do not use the m flag if you use the
rsQuery

command for a serial number.
f filename
Specifies the name of a file that contains vpath names, host volume
names, or volume serial numbers, all of which are to be queried. Use
the f flag parameter with the filename parameter if the q flag with the
volume parameter is not used.
The format of the filename file can be either:
sourceVolume1 targetVolume1
sourceVolume2 targetVolume2
.......................
sourceVolumeN targetVolumeN
or
volume1
volume2
volume3
...........
volumeN

Chapter 7. ESS Command Line Interface
389
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
Example
Example 7-7 shows the output that you might see when you invoke the
rsQuery
command to
query a serial number on an AIX host system.
Example 7-7 Output of rsQuery command
#./rsQuery.sh -v -u yourusername -v yourpassword -q 20223953 -s primaryserverna
************************Volume Information************************
Volume 20223953 found on 23953:12 as volume number 002
Size=2.90_GB
FlashCopy_state=none,
PPRC State=source, Type=extended distance, Status=Extended Distance
PPRCPendingSectors=05486784
PPRC Peer=20224663
Trusted Primed for Resync=no
Session Type: Asynchronous PPRC
Session Number: 50
Session State: Active
******************************************************************
rsQuery: Command successful
The sample output contains the following fields and values:

FlashCopy_state: Specifies the current state of a volume in a FlashCopy relationship with
one of the following values:
– Source
– Target
– None
– Unknown

PPRC State: Specifies the current state of a volume in a PPRC relationship with one of
the following values:
– Simplex
– Source
– Target
– Unknown

Type: Specifies one of the following types of a volume PPRC relationship:
– Synchronous
– Extended Distance

Status: Specifies the current status of a volume in a PPRC relationship with one of the
following values:
– Copy_pending
– Suspended

390
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
– Fullcopy
– None
– Unknown
– Extended Distance

PPRCPendingSectors: Specifies the number of out-of-sync sectors in the relationship at
the time of the last Copy Services refresh. This data may be outdated. To obtain
up-to-date information, use the
rsQueryComplete
command.

Trusted Primed for Resync: Specifies whether the extended distance pair (B to C) in a
cascading PPRC relationship is ready to be resynchronized:
– Yes
– No

Session Type: Specifies the session type as Asynchronous PPRC.

Session Number: Specifies the number of the Asynchronous PPRC session with a value
of 1 - 255.

Session State: Specifies the current volume state in the Asynchronous PPRC
relationship with one of the following values:
– Active
– Join Pending
– Remove Pending
– Unknown State

First Pass: A volume remains in this state until the entire volume has been sent at least
once to the secondary for an initial full volume establish or when all of the out-of-sync
sectors that were indicated in the suspension bit map prior to a resync order have been
sent to the secondary. This state is specified with one of the following values:
– Yes
– No
rsQueryComplete (rsQComp) command
The
rsQueryComplete
command accepts ESS Copy Services server tasks that have been
saved in the ESS Copy Services Web User Interface. It determines whether all volumes
defined in the given tasks have completed the initial PPRC synchronization or FlashCopy
background copy.
When the
rsQueryComplete
command is executed on grouped tasks, the output displays the
first subtask until it reaches the completion threshold, then it displays the next subtask until it
reaches the completion threshold. It repeats this process until all the subtasks are queried.
Syntax of rsQueryComplete command
The syntax for the
rsQueryComplete
command is:
rsQueryComplete.* [-v] {-u username -p password |-a securityfilepath} [-m minutes]
[-t threshold] -s primaryserver [-b backupserver] tasknames
In this command, the asterisk (*) represents the file extension used on the particular
operating system. For further information, refer to Table 7-1 on page 380.
Flags and parameters for rsQueryComplete command
You can specify one or more of the following flags and parameters when you issue the
rsQueryComplete
command:
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.

Chapter 7. ESS Command Line Interface
391
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password-protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
m minutes
Specifies the number of minute and second intervals between queries
for the PPRC or FlashCopy synchronization completion status. The m
flag and the minutes parameter are optional. The default is one
minute.
The format of this parameter is as follows: mm:ss
The mm stands for the number of minutes and ss for the number of
seconds before the command issues a new query.
t threshold
Specifies the threshold percentage that defines the query completion
status. The default is 100%. The t flag and the threshold parameter
are optional.
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
taskNames Specifies the name of one or more previously saved ESS Copy
Services tasks. Separate the task names with a space when you
specify more than one task.

392
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Example
Example 7-8 shows the output that you might see when you invoke the
rsQueryComplete
command and specify the minutes (m) parameter with a value of 0:10 and the threshold (t)
parameter with a value of 80% on an AIX host system.
Example 7-8 Output of rsQueryComplete
#./rsQueryComplete.sh -v -u yourusername -p yourpassword -m 0:10 -t 80
-s primaryservername es_pair_12
rsQueryComplete: Got task manager reference
rsQueryComplete: --------- Task Name: es_pair_12 ---------
rsQueryComplete: Task es_pair_12 found by TaskManager
rsQueryComplete: PPRC Type = Synchronous. Threshold = 80%
rsQueryComplete: waiting 10 seconds...
rsQueryComplete: Sampling volumes...
rsQueryComplete: Percentage complete = 17
rsQueryComplete: The tracks remaining to be copied = 994
rsQueryComplete: waiting 10 seconds...
rsQueryComplete: Sampling volumes...
rsQueryComplete: Percentage complete = 32
rsQueryComplete: The tracks remaining to be copied = 234
rsQueryComplete: waiting 10 seconds...
rsQueryComplete: Sampling volumes...
rsQueryComplete: Percentage complete = 82
rsQueryComplete: The tracks remaining to be copied = 0
rsQueryComplete: ----------------------------------------------
rsQueryComplete: Command successful
The sample output contains one of the following fields and values:

PPRC Type: Specifies one of the following types of PPRC tasks:
– Synchronous
– Extended Distance

FlashCopy: Specifies one of the following types of FlashCopy tasks:
– Background Copy
– No Background Copy
rsTestConnection (rsTConn) command
The
rsTestConnection
command determines whether you can successfully connect to the
ESS Copy Services server.
Syntax of rsTestConnection command
The syntax for the
rsTestConnection
command is:
rsTestConnection.* [-v] -s servername
In this command, the asterisk (*) represents the file extension used on the particular
operating system. For further information, refer to Table 7-1 on page 380.
Flags and parameters for rsTestConnection command
You can specify one or more of the following flags and parameters when you issue the
rsTestConnection
command:
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.

Chapter 7. ESS Command Line Interface
393
s servername
Specifies the IP address or the complete host name of the ESS Copy
Services server to which you want to test the connection.
Example
Example 7-9 shows the output that you might see when you invoke the
rsTestConnection

command on an AIX host system.
Example 7-9 Output or rsTestConnection on an AIX host
#./rsTestConnection.sh -v -s primaryservername
rsWebTest: Using yourhostname as server name
rsWebTest: rsVSServer reference obtained successfully
rsWebTest: rsVSServer reference narrowed successfully
rsWebTest: HeartBeat to the server was successful.
rsWebTest: Command successful
7.2 ESS Storage Management CLI
The Storage Management CLI provides an alternate method to perform ESS logical
configuration and storage management functions. With the Storage Management CLI
(ESSCLI), routine configuration and management tasks can now be automated through their
incorporation into scripts and applications, helping to simplify ESS administration.
The ESS CLI supports the following functions:

Manage one or more ESS servers by reporting about the volumes, volume spaces, I/O
ports, disk groups, and host systems connected to each server

Perform LUN masking by associating or disassociating a volume with a fibre-based host
initiator and by providing volume access information even if the volume is not associated
with an initiator

Manage volume space by querying space availability and creating new volumes within a
volume space

Identify volumes with user-specified labels, if required

List, create, and delete parallel access volumes (PAVs)

Generate an audit log about user ID-specific configuration activities

Define, undefine, and modify a host connection that provides the ESS server with the
information about the host type and worldwide port name

Manage ESS Copy Services tasks by monitoring and querying the tasks in the Copy
Services task repository

Query and report the status of PPRC paths

Manage remote support

Manage Call home

Manage WUI or CLI user names and passwords

List, create, and delete performance data entries
The Storage Management CLI is available for the following operating systems:

IBM AIX

HP-UX

394
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

Linux (Red Hat, RHEL, SUSE LINUX, and SLES)

Sun Solaris

Windows NT® 4.0, Windows 2000, Windows Datacenter, and Windows 2003

OpenVMS Alpha

HP Tru64

IBM NumaQ

Novell Netware
For the latest information about supported operating system versions, refer to the server page
documentation in the inter-operability matrix, which can be found at:
http://www.storage.ibm.com/disk/ess/ess750/supserver.htm
http://www.storage.ibm.com/disk/ess/ess800/supserver.htm
Detailed and complete information on the ESS Copy Services Command Line Interface is
provided in the IBM TotalStorage Enterprise Storage Server Command-Line Interface User’s
Guide, SC26-7494.
7.2.1 Requirements for the Storage Management CLI
The Copy Services Command Line Interface is Java based, and therefore the Java Runtime
Environment needs to be installed on each host system from which you want to issue the
commands.
The use of the Command Line Interface requires, at a minimum, Java JRE level 1.1.8. It is
also supported with Java Level 1.3. For current recommendations, refer to the IBM
TotalStorage Enterprise Storage Server Command-Line Interfaces User’s Guide,
SC26-7494.
On AIX, Windows, SUN Solaris, Linux (Red Hat or SUSE), and OpenVMS, the CLI installation
process automatically installs a JRE on your host system. If you do not accept this option, you
must install the required Java on your host system.
To be able to use the CLI, the host system does not necessarily need to be connected to the
storage subsystem using host ports of the ESS. The only requirement is that the server from
which you want to invoke the CLI commands is connected to the ESS that is defined as the
Primary Copy Services server via a local area network (LAN).
7.2.2 Installing the Storage Management CLI
The Command Line Interface program is located on one of the CDs, which are delivered with
your ESS or with a new LIC version. To install the CLI, you must be logged on to the
respective platform with the highest administrative privilege (root, administrator, or
supervisor).
For specific operating system installation information, refer to the IBM TotalStorage Enterprise
Storage Server Command-Line Interfaces User’s Guide, SC26-7494.
Note: The CLI version discussed is the latest version available at the time of the writing of
this redbook. The CLI version is bound to the LIC level, so if your ESS has an older LIC
level installed, it might be possible that some CLI commands are different or maybe do not
exist.

Chapter 7. ESS Command Line Interface
395
7.2.3 General command guidelines
In general, all command flags, as well as command actions and command categories, can be
specified in any order on the command line. The same applies to the arguments; the
positioning of the name-value pairs is not relevant to the operation of Storage Management
CLI. However, the specification of duplicate command flags and duplicate argument names is
not allowed.
The Command Line Interface is case insensitive, so any combination of upper and lower case
characters will be accepted by the Storage Management CLI for any of the command flags or
argument names, as well as command action and category. For command parameters and
argument values, the case may be significant, depending on the function being performed.
For example, file names are case sensitive in UNIX, of course, and the values of host names
and volume labels are case sensitive in the ESS configuration.
For command parameters and argument values that allow a list of values to be specified, the
list must be comma-separated, without any intervening blanks or other white-space. For
example:
ports = A0,80,04
If a command parameter or argument value contains any embedded blanks, the value must
be surrounded by quotes, either single or double quotes depending on the operating system
being used. For example:
"c:\data files\esscli\ess access file.dat"
"ess=2105.12345 host='Jean Luc' profile=linux"
7.2.4 Copy Services related commands description
The Storage Management CLI provides a single
esscli
command with multiple flags,
parameters, and arguments you can specify.
This section defines the Storage Management CLI commands managing ESS Copy Services
tasks by monitoring and querying the tasks in the Copy Services task repository. This section
also presents the Storage Management CLI command used to monitor and query the PPRC
paths defined for Copy Services. The commands described in this section are:
esscli list task
esscli show task
esscli list PPRCPaths
For further information about configurations and storage management functions, refer to the
IBM TotalStorage Enterprise Storage Server Command-Line Interfaces User’s Guide,
SC26-7494.
esscli list task
This command generates a report that includes all defined tasks in the Copy Services Task
repository.
Syntax of the esscli list task command
The syntax for the
esscli list task
command is:
esscli list task {-a securityfilepath | -p password -u userid} -s primaryserver
[-b backupserver] [-v] [-nohdr]
Flags and parameters for esscli list task command
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.

396
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password-protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
nohdr Omits all header lines from the output.
Example
You might see output similar to Example 7-10 when you invoke the
esscli list task

command on an AIX host system.
Example 7-10 Output of the esscli list task command
#esscli list task -u yourusername -p yourpassword -s primaryservername
Task Name Type Status
------------------------------------------------------------------------
XD_PR_10_11 PPRCEstablishPair NotRunning
FC_PR_11_12 FCEstablish NotRunning
OPEN_SESS_10 OpenCloseSession NotRunning
ADD_VOLS_10 AsyncPPRCStartResume NotRunning
START_ASYNC_10 AsyncPPRCPauseTerminate NotRunning
EstPath1 PPRCEstablishPaths Successful
RemovePaths PPRCRemovePaths Successful
CharliesTask1 PPRCEstablishPair NotRunning
CharliesTask2 PPRCSuspendPair Failed
CharliesTask3 PPRCTerminatePair NotRunning

Chapter 7. ESS Command Line Interface
397
CharliesTask4 PPRCFreezeGroup Successful
CharliesTask5 ConsistencyCreated Successful
MaggiesTask FCWithdraw Successful
RicksTask FCEstablish Successful
aGroupedTask Group Successful
The output contains the following fields and values:

TaskName: The name of the task within the Copy Services task repository.

TaskType: Show the type of the Copy Services task. It gives you basic information about
the task, like if it is FlashCopy Establish task or PPRC terminate task.

Status: Last known status of Copy Services task.
The esscli show task command
This command returns all available information about one or all tasks. It lists attributes of a
task, including source and target volumes. If TaskName is specified, only information for this
task is provided. If omitted, all tasks in the Copy Services Repository are included.
Syntax for esscli show task
The syntax for the
esscli show task
command is:
esscli show task {-a securityfilepath | -p password -u userid} -s primaryserver
[-b backupserver] [-v] [-nohdr] [-d “name=TaskName”]
Flags and parameters for the esscli show task command
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the

398
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
nohdr Omits all header lines from the output.
d Additional argument to specify the task name. If a task name is
specified, only information for this task is provided. The syntax for this
parameter is: -d “name=Taskname”
Example
Example 7-11 shows the output that you might see when you invoke an
esscli show task

command.
Example 7-11 Output of the esscli show tasks command
#esscli show task -v -u yourusername -p yourpassword -s primaryservername -d
"name=MaggiesTask"
Tue May 11 08:23:49 PST 2004 IBM ESSCLI 2.4.0
TaskName=MaggiesTask
TaskType=FCEstablish
Options=TargetOnline,ExtendedDistance,ReadFromSecondary
SourceServer=2105.FA123
TargetServer=2105.FA123
SourceVol TargetVol
--------------------------------------------------------------------------------
1645 1690
1646 1691
1647 692
The sample output contains the following fields and values:

TaskName: The name of the task within the Copy Services task repository.

TaskType: Show the type of the Copy Services task. It gives you basic information about
the task, for example, if it is a FlashCopy Establish task or PPRC terminate task.

Options: This field shows you the options that are used when running a corresponding
task. It gives you information, for example, if the “NoBackground copy” option is used with
the Establish FlashCopy task type.

SourceServer: The ESS ID of the primary storage server. The format of the ESS ID
consists of the machine type, followed by a period (.), followed by the machine sequence
number.

TargetServer: The ESS ID of the secondary storage server. The format of the ESS ID
consists of the machine type, followed by a period (.), followed by the machine sequence
number.

SourceLSS: The logical subsystem on the primary storage server.

TargetLSS: The logical subsystem on the secondary storage server.

SourceVol: The volume ID of the source volume. The first two characters are the LSS
number (00-1F) and the following digits are the volume number (00-FF).

Chapter 7. ESS Command Line Interface
399

TargetVol: The volume ID of the target volume. The first two characters are the LSS
number (00-1F) and the following digits are the volume number (00-FF).
esscli list pprcpaths
Use this command to list the path status for each established PPRC path between a source
and one or more target logical subsystems.
Syntax for esscli list PPRCpaths
The syntax for esscli list PPRCpaths is:
esscli list PPRCpaths {-a securityfilepath | -p password -u userid} -s primaryserver
[-b backupserver] [-v] [-nohdr] [-fmt Format] -d “ess=ESSID [srclss=LssId]”
Flags and parameter for esscli list PPRCpaths
v Displays all responses from the server. This verbose flag is optional
and does not require a parameter.
u username
Specifies the user name that is authorized to execute the Copy
Services CLI commands. The u flag with the username parameter is
required if the administrator for the ESS Copy Services server has
enabled password protection for the host system and if the a flag with
the securityfilepath parameter is not used.
p password
Specifies the password of a user name that is authorized to execute
the Copy Services CLI commands. The p flag with the password
parameter is required if the administrator for ESS Copy Services
server has enabled password protection for the host system and if the
a flag with the securityfilepath parameter is not used.
a securityfilepath
Specifies the full path to the file that contains the user name and
password defined at the ESS Copy Services server for using Copy
Services CLI commands on a host system. The a flag with the
securityfile parameter is required if the administrator for the ESS Copy
Services server has enabled password-protection for the host system
and if the u and p flags and associated parameters are not used.
The format of the record in the security path file is as follows:
username password
s primaryserver
Specifies the IP address or the complete host name of the ESS Copy
Services primary server. Before you use the s flag with the
primaryserver parameter, you must identify and configure an ESS
Copy Services server.
b backupserver
Specifies the IP address or the complete host name of the ESS Copy
Services Backup server. The b flag with the backupserver parameter
is optional. Note that you can use the Backup server only after the
primary server is down.
nohdr Omits all header lines from the output.
fmt Use the fmt flag to define the output fields in the desired order. The
default Format tag is: “srclss,srcport,tgtserver,tgtlss,conn,status”.

400
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
d Additional argument to specify the EssId and the LssID (optional).
The EssId identifies the Enterprise Storage Server (ESS). The format
is “Type.Sequence”, for example, 2105:FA123.
The LssId identifies the logical subsystem in the ESS. The format
consists of two hexadecimal characters, for example, 1F. If specified,
esscli list pprcpaths
will report only those paths where the given
LSS is the source.
Example
Example 7-12 shows the output that you might see when you invoke an
esscli list
pprcpaths
command.
Example 7-12 Output of esscli list pprcpaths command
esscli list pprcpaths -v -u yourusername -p yourpassword -s primaryservername -d
"ess=2105.20288 srclss=16"
Tue May 11 08:23:49 PST 2004 IBM ESSCLI 2.4.0
SourceServer=2105.20288
TotalPaths=7
SrcLss SrcPort TgtServer TgtLss Conn Status
-------------------------------------------------------------------------------
16 80 2105.FA123 18 Fabric 01
16 00 2105.FA123 18 Fabric 01
16 80 2105.FA14 20 Fabric 01
16 00 2105.FA145 20 Fabric 01
16 01 2105.FA178 18 P2P 01
16 04 2105.FA178 18 P2P 01
16 05 2105.FA178 18 P2P 02
The sample output contains the following fields and values:

SourceServer: The ESS ID of the primary storage server. The format of the ESS ID
consists of the machine type, followed by a period (.), followed by the machine sequence
number.

TotalPaths: The total number of paths for the source LSS.

SrcLss: The logical subsystem on the primary storage server.

SrcPort: The port ID on the primary storage server. See Appendix D, “System Adapter ID
(SAID)” on page 555 for details.

TgtServer: The ESS ID of the secondary storage server. The format of the ESS ID
consists of the machine type, followed by a period (.), followed by the machine sequence
number.

TgtLss: The logical subsystem on the secondary storage server.

Conn: PPRC Connection Type:
– Fabric
Path connected through switch
– P2P
Direct connect path

Status: The detailed status of the path (see Table 7-2 on page 401).

TgtPort: The port ID on the secondary storage server. See Appendix D, “System Adapter
ID (SAID)” on page 555 for details (displayed only if requested using the fmt flag).

SwId: The switch identifier (displayed only if requested using the fmt flag).

Chapter 7. ESS Command Line Interface
401

SwPort: The outgoing port on the switch (displayed only if requested using the fmt flag).
Table 7-2 Path status table
7.2.5 Other useful commands
The ESS Storage Management CLI provides other useful commands you can use to manage
various ESS functions from your host system.
We do not provide a full description of those commands in this chapter, since they are not
directly related to the implementation of Copy Services. Nevertheless, we will give a short
description of these commands.
To get a full description of these commands, refer to IBM TotalStorage Enterprise Storage
Server Copy Services Command-Line Interface User’s Guide, SC26-7449.
esscli list server
This command displays the storage server information.
esscli list diskgroup
This command displays information about one or more disk groups. A disk group is a set of
eight logically-associated disk drives that you can configure as a RAID array or as a set of
non-RAID disks. A configured disk group produces one RAID volume space or multiple
non-RAID volume spaces.
Status Description
1 Path established.
2 Initialization failed.
3 Timeout. No reason available.
4 No resources available at primary site for the logical path establishment.
5 No resources available at secondary site for the logical path establishment.
6 Secondary site sequence number or logical subsystem number mismatch.
7 Secondary site SSID mismatch.
8 Path is offline. This is caused by lack of light detection coming from a host, peer, or
switch.
9 Establish failed but will try again when conditions improve.
0A The port at the primary storage server cannot be converted to channel mode because
an inbound logical path is already established.
10 Configuration error.
11 Error during path establishment between fixed-block and CKD logical subsystems.
12 Error during path establish.
13 A Fibre Channel path has been established.
17 The secondary Fibre Channel path is not available.

402
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
esscli list/set port
Displays information about all host adapter ports installed in an ESS server. A host adapter
port enables the attachment of a host to the ESS through a SCSI, ESCON, or Fibre Channel
protocol. The
esscli set port
command modifies the configuration properties of a Fibre
Channel-only host adapter port, such as assigning a topology or protocol to an unconfigured
port.
esscli list log
This command returns an audit log about all the configuration operations that were performed
using the ESS Specialist in the past 10 days.
esscli list/create/delete hostconnection
This set of commands is used to list, define, modify, or remove Fibre Channel host initiators
connected to the ESS.
esscli list/create/delete volumeaccess
This set of commands allows you to list, add, or remove one or more ESS volumes to/from an
attached host system (LUN masking).
esscli list/set/create volume
With this set of commands, you can create or change both fixed-block and CKD volumes. You
can also list information about the volumes.
esscli list/create/delete volumespace
You can use this set of commands to create or delete configured storage space. A volume
space represents a contiguous space from which storage volumes are allocated. For the
ESS, a volume space is a RAID array or a non-RAID (JBOD) disk. When a disk group is
configured as a RAID array, one volume space is created. When a disk group is configured as
non-RAID array, up to eight volume spaces are created, one for each disk in the group.
The command will return before the actual volume space is formatted. A delete and create
command cannot be run immediately in sequence on the same disk group. The user should
poll the ESS to see if the volume space has finished formatting.
esscli list featurecode
Lists the active feature codes on the ESS. The Licensed Feature Codes table displays the
licensed feature codes that you have purchased for this ESS. Feature codes enable the use
of optional software components that provide additional or enhanced ESS functions.
esscli list/create/set/delete webuseraccount
This new set of commands helps to manage the user accounts on the ESS. You can use the
commands to:

List the active users on the ESS.

View the settings of the currently defined user accounts that are permitted to access the
ESS.

Create new user accounts that can access the ESS.

Modify Web user accounts that can access the ESS.

Change user names.

Change passwords.

Chapter 7. ESS Command Line Interface
403

Modify access levels.

Modify IP ranges.

Delete users.
Most of these commands require Administration access level.
esscli create/delete/list perfstats
Use this set of commands to display and modify the current settings of the data collector.
These settings include the name of the machine that is to receive the collected statistics, the
frequency at which statistics should be collected, the port number where the statistics should
be sent, and the status of the data collection.
The
esscli create perfstats
command has a new option to list Asynchronous PPRC
information, which comes with PPRC V2 and LIC level 2.4.x. For more information, refer to
IBM TotalStorage Enterprise Storage Server Copy Services Command-Line Interface User’s
Guide, SC26-7449.
Example 7-13 shows the output that you might see when you invoke an
esscli create
perfstats
command with the async=on option.
Example 7-13 Output of the esscli create perfstats command with async=on option
esscli -u user -p password -s server create PerfStats -d "ess=2105.12345 outputdir=.
frequency=11 duration=1 async=on"
Machine:
machinename.ibm.com
Date/Time Fail Success % Success
Wed May-12-04 15:03:10 delta: 0 2 100
cum: 0 294 100
RPO Coord. Time Interval Time Max Drain Time
5.000 (sec) 50 (ms) 5 (sec) 30 (sec)
Time since last successful consistency group: 2 (sec)
The sample output contains the following fields and values:

Date/Time: Date and time from the ESS

Fail: Number of Consistency Groups that did not complete in the user specified drain time

Success: Number of Consistency Groups that completed in the user specified drain time

% Success: Percentage of successfully completed Consistency Groups

cum: Cumulative values

delta: Difference between the current cumulative values and the previous cumulative
values (previous cumulative values were some time ago, while the time is set by the
frequency parameter)

RPO: (current Date/Time - previous Date/Time) / delta number of successful Consistency
Groups

Coord. Time: Maximum Coordination Timer (for more information, see 4.5,
“Asynchronous PPRC” on page 74)

Interval Time: Consistency Group Timer (for more information, see 4.5, “Asynchronous
PPRC” on page 74)

404
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

Max Drain Time: Maximum Consistency Group Drain Time (for more information, see 4.5,
“Asynchronous PPRC” on page 74)
esscli show/set remotesupport
Display the current settings set for the ESS remote support. With this set of commands, you
can:

Indicate whether incoming and outgoing calls are enabled.

Modify or set the remote support properties available in the ESS.

Set the call home and remote access properties.

Set service and PE passwords.
esscli show/create/delete/set email
This set of commands will let you manage all the information regarding the e-mail addresses
configured for problem notification. You can:

Display the information about e-mail addresses.

The output contains the e-mail addresses and the corresponding error notification trigger
value indicating the types of problems.

Add a new e-mail address to the problem notification configuration.

Remove an e-mail address from the current problem notification configuration.

Modify an existing e-mail address and its corresponding options.

Change the trigger options on an existing e-mail address.

Rename the existing e-mail address with a new address.

Change the global settings of e-mail.

Set the maximum number of e-mails sent per problem.

Send test e-mail.
esscli show/create/delete/set pager
You can manage the pager settings configured for problem notification with the CLI
commands. With these commands, you can:

Display a list of all pagers, with all their related information, and global pager settings.

Add a new pager number to the problem notification configuration.

Remove a pager from the current problem notification configuration.

Modify existing pager information.

Update the global properties of pagers.
esscli show/create/delete/set snmp
This set of commands will let you manage all the information regarding the snmp
configuration for problem notification. You can:

Add trap addresses to the existing problem notification configuration.

Modify existing SNMP information.

Delete trap addresses from the SNMP problem notification configuration.

Chapter 7. ESS Command Line Interface
405
esscli list/show/delete/ problem
You can use these commands to manage the active problems on an ESS. With these CLI
commands, you can:

List all outstanding problems active on the ESS (this will output a table that displays the
problem ID, cluster, and description of each active problem).

Get information on specific active problems.

View the currently active problem log records (when an ESS monitors the operation of its
internal components and logical resources, abnormal conditions are entered in the
problem log).

Cancel any outstanding problems active on the ESS.
esscli list/create/delete pav
You can use this set of commands to manage Parallel Access Volumes. It allows you to:

Configure one or more PAVs for a CKD base volume. The total number of PAVs and base
volumes in an LSS cannot exceed 256.

Display information about PAVs configured for a particular logical subsystem or logical
volume in an ESS.

Remove one or more existing parallel access volumes (PAVs) for a CKD base volume.

406
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
407
Chapter 8.
Open systems specifics
In this chapter, we describe the basic tasks that need to be performed on the individual host
systems when using ESS Copy Services.
We explain how to bring FlashCopy target volumes online to the same host as well as to a
second host. The chapter covers various UNIX and WinTel platforms.
8

408
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
8.1 AIX specifics
In this section, we describe the steps needed to use volumes created by the ESS Copy
Services on AIX hosts.
8.1.1 AIX and FlashCopy
The FlashCopy functionality from the ESS Copy Services copies the entire contents of a
source volume to a target volume. If the source volume is defined to the AIX Logical Volume
Manager (LVM), all of its data structures and identifiers are copied to the target volume, as
well. This includes the Volume Group Descriptor Area (VGDA), which contains the Physical
Volume Identifier (PVID) and Volume Group Identifier (VGID).
For AIX LVM, it is currently not possible to activate a volume group with a physical volume
(hdisk) that contains a VGID and a PVID that is already used in a volume group existing on
the same server. The restriction still applies even if the hdisk PVID is cleared and reassigned
with the two commands listed in Example 8-1.
Example 8-1 Clearing PVIDs
chdev -l <hdisk#> -a pv=clear
chdev -l <hdisk#> -a pv=yes
Therefore, it is necessary to redefine the volume group information on the FlashCopy target
volumes using special procedures or the
recreatevg
command (refer to “AIX recreatevg
command” on page 410). This will alter the PVIDs and VGIDs in all the VGDAs of the
FlashCopy target volumes, so that there are no conflicts with existing PVIDs and VGIDs on
existing volume groups that reside on the source volumes. If you do not redefine the volume
group information prior to importing the volume group, then the
importvg
command will fail.
Accessing FlashCopy target volume from another AIX host
The following procedure makes the data of the FlashCopy target volume available to another
AIX host that has no prior definitions of the target volume in its configuration database (ODM):
1.The target volume (hdisk) is new to AIX, and therefore the Configuration Manager should
be run on the specific SCSI or Fibre Channel adapter:
cfgmgr -l <host_bus_adapter>
2.Find out which of the physical volumes is your FlashCopy target volume:
lsdev -Cc disk | grep 2105
3.Import the target volume group:
importvg -y <volume_group_name> <hdisk#>
4.Vary on the Volume Group (the
importvg
command should varyon the volume group):
varyonvg <volume_group_name>
5.Verify consistency of all file systems on the FlashCopy target volume:
fsck -y <filesystem_name>
6.Mount all the target file systems:
mount <filesystem_name>
The data is now available. You can, for example, back up the data residing on the FlashCopy
target volume to a tape device. This procedure can be run once the relationship between the
FlashCopy source and target volume is established, even if data is still being copied in the
background.

Chapter 8. Open systems specifics
409
The disks containing the target volumes may have been previously defined to an AIX system,
for example, if you periodically create backups using the same set of volumes. In this case,
there are two possible scenarios:

If no volume group, file system, or logical volume structure changes were made, then use
Procedure 1 to access the FlashCopy target volumes from the target system.

If some modifications to the structure of the volume group were made, such as changing
the file system size or the modification of logical volumes (
LV
), then it is recommended to
use Procedure 2 and not Procedure 1.
Procedure 1
1.Unmount all the source file systems:
umount <src_filesystem>
2.Unmount all the target file systems:
umount <tgt_filesystem>
3.Deactivate the target volume group:
varyoffvg <tgt_volume_group_name>
4.Establish the FlashCopy relationship(s).
5.Mount all the source file systems:
mount <src_filesystem>
6.Activate the target volume group:
varyonvg <tgt_volume_group_name>
7.Perform a file system consistency check on target file systems:
fsck -y <tgt_filesystem>
8.Mount all the target file systems:
mount <tgt_filesystem>
Procedure 2
1.Unmount all the target file systems:
umount <tgt_filesystem>
2.Deactivate the target volume group:
varyoffvg <tgt_volume_group_name>
3.Export the target volume group:
exportvg <tgt_volume_group_name>
4.Delete the target physical volumes (optional; not needed if using the same set of physical
volumes):
rmdev -dl <hdisk#>
5.Establish the FlashCopy relationship(s).
6.Then perform tasks as if the volumes were new to the system (if you did not remove the
physical volumes from the target systems, then you do not have to run the Configuration
Manager or identify the volumes).
Accessing the FlashCopy target volume from the same AIX host
In this section, we describe a method of accessing the FlashCopy target volume on a single
AIX host while the source volume is still active on the same server. The procedure is intended
to be used as a guide and may not cover all scenarios.

410
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
The steps needed to access the FlashCopy target volume depend on the level of AIX used. If
your level is AIX Version 4.3.3 maintenance level 05 (APAR IY10456) or higher, use the
procedure outlined in “AIX recreatevg command” on page 410.
AIX recreatevg command
Copying the source volume's content using FlashCopy causes all of the data structures and
identifiers used by AIX's Logical Volume Manager to be duplicated to the target volume. The
duplicate definitions (PVID and VGID) in turn cause conflicts within LVM. This problem is
solved by using the AIX command
recreatevg
.
The
recreatevg
command is packaged as a PTF for AIX Version 4.3.3 in APAR IY10456 and
higher. It is officially available in:

AIX 4.3.3 Recommended Maintenance Level 05 (RML05) or higher

AIX 5L Version 5.1

AIX 5L Version 5.2
The
recreatevg
command overcomes the problem of duplicated LVM data structures and
identifiers caused by a disk duplication process such as FlashCopy. It is used to recreate an
AIX Volume Group (VG) on a set of target volumes that are copied from a set of source
volumes belonging to a specific VG. The command will allocate new physical volume
identifiers (PVIDs) for the member disks and a new volume group identifier (VGID) to the
volume group. The command also provides options to rename the logical volumes with a
prefix you specify, and options to rename labels to specify different mount points for file
systems.
Here is the AIX man page synopsis (might be different, depends on the AIX version used):
recreatevg [ -y VGname ] [ -p ] [ -f ] [ -Y lv_prefix | -l LvNameFile ] [ -L label_prefix ]
[ -n ] \ PVname...

Description
This command can be used to recreate a VG on a set of disks that are mirrored from a set
of disks belonging to a specific VG. This command will allocate new physical volume
identifiers (PVID) for the member disks, as the PVIDs will also be duplicated by the disk
mirroring. Similarly, other LVM logical members that are duplicated will also be changed to
new names with the specified prefixes.

Flags
-y VolumeGroup specifies the volume group name rather than having
the name generated automatically. Volume group names must be
unique system wide and can range from 1 to 15 characters. The
name cannot begin with a prefix already defined in the PdDv class
in the Device Configuration database for other devices. The volume
group name created is sent to standard output.
-p Disables the automatic generation of the new PVIDs. If the -p flag
is used, you must ensure that there are no duplicated PVIDs on the
system. All the disks that were hardware mirrored must have had
their PVIDs changed to an unique value.
-Y lv_prefix causes the logical volumes on the VG being recreated
renamed with this prefix. The number of characters in the prefix
should be such that the total length of the prefix and the logical
volume name must be less than or equal to 15 characters. If the
length exceeds 15 characters, the logical volume will be renamed
with the default name. The name cannot begin with a prefix already

Chapter 8. Open systems specifics
411
defined in the PdDv class in the Device Configuration Database for
other devices, nor be a name already used by another device.
-L label_prefix causes the labels of logical volumes on the VG being
recreated to be changed with this prefix. The user must modify the
/etc/filesystems stanza manually if a simple modification of the
mount point is not enough to define the stanza uniquely.
-l LvNameFile entries in the LvNameFile must be in the format
LV1:NEWLV1. After
recreatevg
, LV1 will be renamed with
NEWLV1. All the logical volumes that are not included in the
LvNameFile will be recreated with the default system generated
name.
-f Allows a volume group to be recreated that does not have all disks
available.
-n After
recreatevg
, the volume group is imported but varied off. The
default is imported and vary on.

Notes
a.To use this command, you must have root user authority.
b.All the member physical volumes of the volume group must be specified on the
command line. The command will fail if the input list does not match with the list
compiled from the Volume Group Descriptor Area (VGDA).
c.If you perform a Copy Services function on one half of a RAID-1 pair to reduce the
capacity required for FlashCopy targets or PPRC secondary volumes, then use the -f
option to force the creation of the volume group. Otherwise, the VGDA will have PVIDs
of volumes that made up the other half of the mirror at the source or primary site.

Examples:
a.To recreate a volume group that contains three physical volumes, enter this command:
recreatevg hdisk1 hdisk2 hdisk3
The volume group on hdisk1, hdisk2, and hdisk3 is recreated with an automatically
generated name, which is displayed.
b.Run:
recreatevg -y testvg hdisk1
The volume group on hdisk1 is recreated with the new name testvg.
c.Run:
recreatevg -Y newlv hdisk14
The volume group on hdisk14 is recreated and all logical volumes in that volume group
are recreated and renamed with the prefix newlv.
Accessing FlashCopy target volume using the recreatevg command
In this example, we have a volume group containing two physical volumes (hdisks) and wish
to FlashCopy the volumes for the purpose of creating a backup.
The source volume group is fc_source_vg, containing hdisk4 and hdisk5.
The target volume group will be fc_target_vg, containing hdisk8 and hdisk9.

412
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Perform these tasks to run the FlashCopy and make the target volumes available to AIX:
1.Stop all applications that access the FlashCopy source volumes.
2.Unmount all source file systems for the short period of FlashCopy establishment.
3.Establish the FlashCopy pairs with the No Background Copy option selected. Use the
ESS Copy Services Web user interface (WUI) to establish the pairs or, if you have a task
defined, use
rsExecuteTask.sh
in the CLI.
4.Mount all source file systems.
5.Restart applications that access the FlashCopy source volumes.
6.The target volumes, hdisk8 and hdisk9, will now have the same volume group data
structures as the source volumes hdisk4 and hdisk5. Clear the PVIDs from the target
hdisks to allow a new volume group to be made:
chdev -l hdisk8 -a pv=clear
chdev -l hdisk9 -a pv=clear
The output from
lspv
shows the result (see Figure 8-1).
Figure 8-1 lspv after pv=clear
7.Create the target volume group and prefix all file system path names with /backup and
prefix all AIX logical volumes with bkup:
recreatevg -y fc_target_vg -L /backup -Y bkup hdisk8 hdisk9
You must specify the hdisk names of all disk volumes participating in the volume group.
The output from
lspv
illustrates the new volume group definition (see Figure 8-2).
Figure 8-2 Recreated FlashCopy target volumes
An extract from /etc/filesystems shows how
recreatevg
generates a new file system stanza.
The file system named /u01 in the source volume group is renamed to /backup/u01 in the
target volume group. Also, the directory /backup/u01 is created. Notice also that the logical
volume and JFS log logical volume have been renamed. The remainder of the stanza is the
same as the stanza for /u01 (see Figure 8-4 on page 416).
# lspv
hdisk4 000567992d4c9024 fc_source_vg
hdisk5 000567995abe005e fc_source_vg
hdisk8 none None
hdisk9 none None
# lspv
hdisk4 000567992d4c9024 fc_source_vg
hdisk5 000567995abe005e fc_source_vg
hdisk8 000567995abdf345 fc_target_vg
hdisk9 00056799b2d831b9 fc_target_vg

Chapter 8. Open systems specifics
413
Figure 8-3 Target file system stanza
8.Mount the new file systems belonging to the target volume group to make them
accessible.
More detail is given on using AIX and FlashCopy as part of your backup strategy in
Appendix F, “AIX and FlashCopy for backups” on page 571.
8.1.2 AIX and PPRC
When you have the primary and secondary volumes in a PPRC relationship, it is not possible
to read the secondary unless the Permit read from secondary option has been selected
when establishing the relationship (see 6.14, “Establishing a Synchronous PPRC pair” on
page 266). To be able to read the secondary volumes, they must also be in the full duplex
state (in addition to Permit read from secondary option). Therefore, if you are configuring
the secondary volumes on the target server, it is necessary to terminate the copy pair
relationship. Once the volumes are in the simplex state, the secondary volumes can be
configured (
cfgmgr
) into the target systems customized device class (CuDv) of the ODM. This
will bring in the secondary volumes as hdisks and will contain the same physical volume IDs
(PVID) as the primary volumes. Because these volumes are new to the system, there is no
conflict with existing PVIDs. The volume group on the secondary volumes containing the
logical volume (LV) and file system information can now be imported into the Object Data
Manager (ODM) and the /etc/filesystems file using the
importvg
command.
If the PPRC secondary volumes were previously defined on the target AIX system as hdisks
or vpaths, but the original volume group was removed from the primary volumes, the old
volume group and disk definitions must be removed (
exportvg
and
rmdev
) from the target
volumes and redefined (
cfgmgr
) before running
importvg
again to get the new volume group
definitions. If this is not done first,
importvg
will import the volume group improperly. The
volume group data structures (PVIDs and VGID) in ODM will differ from the data
structures in the VGDAs and disk volume super blocks. The file systems will not be
accessible.
If the secondary volumes that are already configured on the target AIX server are in a PPRC
relationship and you do not have the Permit read from secondary option enabled (and the
volumes are not in full duplex state), after rebooting the target server, the hdisks will be
configured to AIX again. In other words, you will see each PPRC secondary volume twice on
the target server. The reason for this situation is as follows: AIX knows that these physical
volumes already exist with entries in the Configuration Database (ODM). However, when the
configuration manager runs during reboot, it cannot read their PVIDs because, as PPRC
targets, they are locked by the ESS Copy Services server. This results in AIX causing the
original hdisks to be configured to a Defined state, and new (phantom) hdisks being
configured and placed in an Available state. This is an undesirable condition that must be
remedied before the secondary volumes can be accessed.
/backup/u01:
dev = /dev/bkupelv001
vfs = jfs
log = /dev/bkupelvlog001
mount = true
check = false
options = rw
account = false

414
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
To access the secondary volumes, the phantom hdisks must be removed and the real or
original hdisks must be changed from a Defined state to an Available state.
For example, hdisk6 through hdisk9 are assigned to a volume group, evg001. Each of the disk
volumes is currently participating as a secondary volume in a PPRC relationship. If the server
is rebooted, four new hdisks are configured to AIX. These phantom disks, hdisk13 through
hdisk16, appear in the output from
lspv
, as shown in Figure 8-4.
Figure 8-4 PPRC phantom disks
When you execute
lsdev -Cc disk
, you can observe that the state of the original PPRC
secondary volumes has become Defined during reboot (shown in Figure 8-5).
Figure 8-5 Phantom hdisks
It is important to execute both the
lspv
and
lsdev
commands, so that you can be sure which
disks are the phantoms. From the
lspv
output, the phantom disks will have no PVIDs and will
not be assigned to a volume group. From the
lsdev
output, the phantom will be in an Available
state. The original disks will have PVIDs assigned to a volume group and will be in a Defined
state.
To remove the phantom hdisks from the configuration database, run the
rmdev -dl
command
on each phantom disk device (see Example 8-2).
Example 8-2 Removing phantom hdisks
# for i in 13 14 15 16
do
rmdev -dl hdisk$i
done
Set the original hdisks to an Available state with the
mkdev
command (see Example 8-3 on
page 415).
# lspv
hdisk6 000567992d4c9024 evg001
hdisk7 000567995abe005e evg001
hdisk8 000567995abdf345 evg001
hdisk9 00056799b2d831b9 evg001
hdisk13 none None
hdisk14 none None
hdisk15 none None
hdisk16 none None
# lsdev -Cc disk
hdisk6 Defined 20-58-01 IBM FC 2105F20
hdisk7 Defined 20-58-01 IBM FC 2105F20
hdisk8 Defined 20-58-01 IBM FC 2105F20
hdisk9 Defined 20-58-01 IBM FC 2105F20
hdisk13 Available 20-58-01 IBM FC 2105F20
hdisk14 Available 20-58-01 IBM FC 2105F20
hdisk15 Available 20-58-01 IBM FC 2105F20
hdisk16 Available 20-58-01 IBM FC 2105F20

Chapter 8. Open systems specifics
415
Example 8-3 Configuring hdisks
# for i in 6 7 8 9
do
mkdev -l hdisk$i
done
You can reactivate the volume group, evg001, (
varyonvg
) and mount its file systems.
Making updates to the LVM information
When performing PPRC between primary and secondary volumes, the primary AIX host may
create/modify or delete existing LVM information from a volume group. However, because the
secondary volume is not accessible when in a PPRC relationship, the LVM information in the
secondary AIX host would be out-of-date. Therefore, scheduled periods need to be allotted
where write I/Os to the primary PPRC volume can be quiesced and file systems unmounted.
At this point, the copy pair relationship can be terminated and the secondary AIX host can
perform a learn on the volume group (
importvg -L
).
Once the updates have been imported into the secondary AIX hosts ODM, you can establish
the PPRC copy pair again. However, select Do not copy volume from the Select copy
Options when establishing the PPRC copy pair. As soon as the PPRC pair has been
established, immediately suspend the PPRC relationship. Because there was no write I/O to
the primary volumes, both primary and secondary are consistent.
Now that the primary volume has been suspended, the file systems can be remounted and
write I/O resumed. Once the write I/O has been going for a while, you can reestablish the
relationship with the primary and secondary by choosing Copy out-of-sync cylinders only

(see 6.15.7, “Resynchronizing PPRC copy pairs” on page 283).
If the Permit read from secondary option was selected during the PPRC copy pair establish,
then it would be advisable to suspend the primary volume (while in the full duplex state) and
then perform the import learn function. Once completed, all that is necessary would be to
reestablish the copy pair only by copying the out-of-sync cylinders.
The following example shows two systems, sanf50 and sanh70, where sanf50 has the
primary volume vpath5 and sanh70 has the secondary volume vpath16. Both systems have
had their ODMs populated with the volume group itsovg from their respective PPRC volumes
and, prior to any modifications, both systems ODM have the same time stamp, as shown in
Figure 8-6.
Figure 8-6 Original time stamp
Volumes vpath5 and vpath16 are in the PPRC duplex state, and the volume group itsovg on
sanf50 is updated with a new logical volume. The time stamp on the VGDA of the volumes
gets updated and so does the ODM on sanf50, but not on sanh70 (see Figure 8-7 on
page 416).
root@sanf50:/ > getlvodm -T itsovg
3d99d8911542ab68
root@sanh70:/ > getlvodm -T itsovg
3d99d8911542ab68

416
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-7 Updated source time stamp
To update the ODM on the secondary server, it is advisable to suspend the PPRC copy pair
prior to performing the
importvg -L
command to avoid any conflicts from LVM actions
occurring on the primary server. Figure 8-8 shows the updated ODM entry on sanh70.
Figure 8-8 Update secondary server’s ODM
Once the
importvg -L
command has completed, you can reestablish the PPRC copy pairs
and copy only the out-of-sync cylinders.
8.1.3 Scripting Copy Services tasks
The following scripts were created and run in the lab. Notice that they are shown as examples
only and will need modification to run in your environment.
Automation of consistent backup using PPRC-XD
This script uses the following ESS Copy Services tasks when calling
rsExecuteTask.sh

commands.
The PPRC_Est_Susp task will transform the asynchronous PPRC-XD relationship into a
Synchronous PPRC relationship and suspend the pair once they are synchronized (see
Example 8-4).
Example 8-4 PPRC_Est_Susp task
Task Information
---------------------------------------------------------------
Task(C)
Task type: Establish Synchronous PPRC pair
Task options: Copy out-of-sync cylinders only
Permit read from secondary
Suspend PPRC after establish
Source: Target:
18767:14 12320:14
root@sanf50:/ > lqueryvg -p vpath5 -Tt
Time Stamp: 3d99e5de077392e4
root@sanf50:/ > getlvodm -T itsovg
3d99e5de077392e4
root@sanh70:/ > lqueryvg -p vpath16 -Tt
Time Stamp: 3d99e5de077392e4
root@sanh70:/ > getlvodm -T itsovg
3d99d8911542ab68
root@sanh70:/ > importvg -L itsovg vpath16
root@sanh70:/ > lqueryvg -p vpath16 -Tt
3d99e5de077392e4
root@sanh70:/ > getlvodm -T itsovg
3d99e5de077392e4

Chapter 8. Open systems specifics
417
volume: 002 (40218767) volume: 000 (40012320)
The FlashCopy_1 task creates a FlashCopy of the PPRC secondary volume (40012320)
with the background copy option, which can be used to back up the data (see Example 8-5).
Example 8-5 FlashCopy_1 task
Task Information
---------------------------------------------------------------
Task(FlashCopy_1)
Task type: FlashCopy establish
Task options: none
Source: Target:
12320:14 12320:14
volume: 000 (40012320) volume: 001 (4012320)
The PPRC-XD_Est_OS task is used to reestablish the PPRC-XD relationship as it was
before the backup process began by establishing the PPRC-XD relationship and copying
out-of-sync cylinders (see Example 8-6).
Example 8-6 PPRC-XD_Est_OS task
Task Information
---------------------------------------------------------------
Task(PPRC-XD_Est_OS)
Task type: Establish extended distance PPRC pair
Task options: Copy out-of-sync cylinders only
Permit read from secondary

Source: Target:
18767:14 12320:14
volume: 002 (40218767) volume: 000 (40012320)
The consbkup.sh script can be used to perform a consistent backup of the production data by
stopping application IO, changing the PPRC-XD pair into a synchronized PPRC pair,
suspending the pair, running a FlashCopy of the PPRC secondary volume to create tertiary
volume, backing up the tertiary volume, restarting the IO, and then returning the PPRC pair
back to an asynchronous PPRC-XD pair. The script will require modification to work properly.
It is intended to provide the steps that must be performed to accomplish the mentioned goals.
It is just a simple example without any testing of prerequisites and error condition handling
(see Example 8-7).
Example 8-7 consbkup.sh
#!/bin/sh
#
#
#Stop all applications and unmount filesystems on source
umount /fs1
umount /fs2
#

418
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
#Synchronize the PPRC-XD pairs and then suspend
rsExecuteTask.sh -u username -p password -s 9.88.9.1 -b 9.88.9.2 PPRC_Est_Susp
#
#FlashCopy the PPRC target
rsExecuteTask.sh -u username -p password -s 9.88.9.1 -b 9.88.9.2 FlashCopy_1
#
#Backup the data
tar -cvf files backupfiles
#
#Restart application and mount filesystems
mount /fs1
mount /fs2
#
#reestablish the PPRC-XD relationship and copy out-of-sync cylinders
rsExecuteTask.sh -u username -p password -s 9.88.9.1 -b 9.88.9.2 PPRC-XD_Est_OS
8.2 Windows NT specifics
In this section, we discuss what tasks are necessary when performing Copy Services
operations on volumes owned by Microsoft Windows NT hosts:

8.2.1, “Windows NT basic volumes” on page 418 covers volumes not involved in volume
sets, mirrors, stripe, or RAID sets.

8.2.2, “Copy Services with Windows NT volume sets” on page 420 discusses operations
with Windows NT volume sets.
Specifics of Windows 2000 hosts will be explained below in 8.3, “Windows 2000 and Copy
Services” on page 423.
8.2.1 Windows NT basic volumes
Windows NT handles disks in a way that is not similar to any other operating system covered
in this book. The need to reboot a server in order to scan for new disks and the need to run a
GUI-based Disk Administrator in order to manipulate the disks are the main factors that
restrict the routine use of PPRC or FlashCopy.
It is possible to automate the actions of the GUI-based Disk Administrator using third-party
software to remotely reboot the server and to remotely assign the drive letter from the server
that starts the Copy Services task. This was not tested during our project. You can automate
the invocation of PPRC and FlashCopy using the ESS Copy Services Command Line
Interface (CLI).
If you are going to create an automated script with Windows NT, you need to be very careful
about data consistency. It may be that some part of the automation process may run a script
on a source server and subsequent actions maybe taken by a script on a target server.
Therefore, inter process communication across servers may be required for timing or you may
get inconsistent data (not all applications will allow this).
You have two options on how to make a PPRC or FlashCopy target available to the server,
with reboot
or
without reboot
. We recommend that you reboot the server; it is safer because
then it is guaranteed that all the registry entries get created. However, using PPRC and
FlashCopy without rebooting is faster.
Note: When calling Copy Services tasks inside shell scripts or at the command line, be
aware that the task names are
case-sensitive
.

Chapter 8. Open systems specifics
419
Registering PPRC/FlashCopy volumes to NT
If you are going to reboot the server, you do not have to make the target disks known to
Windows NT before you do the PPRC or FlashCopy. However, we recommend that you have
them preassigned and registered in the server. The “assign disk and run PPRC/FlashCopy”
approach is useful for a non-routine PPRC/FlashCopy, for example, for testing or migration.
For routine purposes, we recommend having target disks already present in the Disk
Administrator with partitions created and partition information saved. You can accomplish that
by clicking Start

Programs

Administrative Tools

Disk Administrator:
1.If the target disk has not been previously seen by the system, Disk Administrator will issue
a pop-up message saying “No signature on Disk X. Should I write a signature?” where X is
the number assigned to the newly present disk.
2.Click OK to save the signature on the target disk. The Disk Administrator will come up.
3.Click on the disk that is to be used as the PPRC/FlashCopy target (it should be grey and
marked as Free Space) and select Create.
4.Confirm the partition parameters and click OK. The partition appears as Unknown.
5.Click on the newly created partition and select Commit Changes Now.
6.Right-click on the partition and select Assign Drive letter.
7.Assign a drive letter and click OK. Exit Disk Administrator.
After this procedure, the information about the PPRC/FlashCopy target volume is properly
stored in the Windows NT registry.
Stopping the target server
Stop the server that will use the target if you want to use the safer method. Also bear in mind
that if you were assigning the volume to the host just before performing the PPRC/FlashCopy,
you will have to use the volume serial number for the target. You cannot use the
rsPrimeServer
CLI command to insert disk names into the ESS Copy Services Web
Interface, as the server is down.
Performing a FlashCopy
Stop all applications using the source volume. Now you must flush the data to the source
volume. You can accomplish that by clicking Start

Programs

Administrative Tools

Disk Administrator.
1.Click on the disk that is to be used as the PPRC/FlashCopy source (it should have a drive
letter assigned and be formatted) and select Assign Drive letter.
2.In the pop-up box, select Do not assign a drive letter and click OK.
Now the data is flushed to the source and you can start the PPRC/FlashCopy task from
the ESS Copy Services Web Interface or from any server CLI.
Observe the GUI or issue the
rsQuery
command to the volumes to see if the
PPRC/FlashCopy task had successfully started. Then you can reassign the drive letter to
the source volume.
3.Click on the disk that is a PPRC/FlashCopy source and select Assign Drive Letter.
4.Assign a drive letter and click OK. Exit Disk Administrator.
You can resume using the source volume.

420
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Starting the target server
After that, you may boot up the target server. In this case, you have just assigned the target
volumes to the host that will create the disk entry in the Windows NT registry. To verify that
the registry entry is created, do the following:
1.Click Start

Settings

Control Panel.
2.In Control Panel, double-click SCSI Adapters.
3.Click the adapter that has the target volume attached.
That opens a list of targets. Verify the list includes the target ID and LUN of the volume you
have just made available to the server. If you are using SDD, you will see each disk entry
several times, depending on how many paths to a volume you have defined.
You may also run the command
datapath query device
from the SDD command line to
check if the PPRC/FlashCopy targets are listed between the volumes. This command will also
enable you to check volume serial numbers, and will give you a more understandable
overview of the volumes and their paths.
Making PPRC/FlashCopy target available
Log in, start Windows NT Disk Administrator, write a signature if necessary, and assign a
drive letter. To do that, click Start

Programs

Administrative Tools

Disk
Administrator.
1.If the disk has not been previously seen by this system, Disk Administrator will issue a
pop-up message saying “No signature on Disk X. Should I write a signature?”, where X is
the number assigned to the newly present disk.
2.Click OK to save the signature on the target disk. The Disk Administrator will come up.
3.Click on the disk that is a PPRC/FlashCopy target (you should see a formatted partition on
it) and select Assign Drive Letter.
4.If you cannot assign a drive letter, the target is probably corrupt. Try repeating the whole
process and consider the scenario that includes reboot.
5.Assign a drive letter and click OK. Exit Disk Administrator.
6.From a Windows NT command prompt, run
chkdsk x: /f /r
, where x is the letter
assigned to the PPRC/FlashCopy target. An option is to run the disk check from the
Properties of a disk in Windows NT Explorer.
7.After this procedure, the PPRC/FlashCopy target is available to the Windows NT and can
be handled like normal disk.
8.2.2 Copy Services with Windows NT volume sets
The following section describes how to perform Copy Services functions with Windows NT
volume sets. Basically, the same procedures apply to other types of Windows NT
multi-volume logical drives, such as mirrored sets, striped sets, and striped sets with parity.
But in practice, because the ESS provides RAID-1+0 and RAID-5, only volume sets are
needed (to add more space to existing logical drives).
In Windows NT, information about multi-volume logical drives is stored only in the registry key
HKEY_LOCAL_MACHINE\System\Disk (shortly called the Disk key). Because there is no
volume information on the disks itself, it is not possible to import a multi-volume logical drive
by another host. Instead, the Disk key of that host must be changed. Unfortunately, the Disk
key value is in binary format, requiring special tools for manipulations.

Chapter 8. Open systems specifics
421
You can find information about the Disk key’s internal structure at the Web site:
http://www.sysinternals.com/ntw2k/info/diskkey.shtml
Both PPRC and FlashCopy are supported when using normal disks and volume sets. When
using either PPRC or FlashCopy with volume sets, because these outboard copy features do
not copy the volume set information in the Windows registry, certain limitations exist, and a
special procedure is required, as outlined below. After SP6, it is possible to have the
FlashCopy source and target volumes accessible by the same server. Prior to SP6, the
FlashCopy source and target volumes must be attached to different servers. PPRC primary
and secondary volumes must always be attached to different servers.
Procedure for using PPRC and FlashCopy with volume sets
This special procedure is required in order to FlashCopy or PPRC an NT volume set. If the
target disks are in the same order as the source disks,
and
the target disks are contiguous
(that is, all the disks are next to each other, as viewed by the target machine’s Disk
Administrator), then simply create an identical volume set on the target machine and reboot
prior to performing the FlashCopy. This only has to be done before performing FlashCopy or
PPRC for the first time. Subsequent copies should work as expected, provided that the file
system is unmounted (the drive letter is unassigned) on the target prior to performing a copy.
If the target disks do not appear contiguous to NT or appear in a different order than on the
source machine, then a different procedure must be used. Microsoft’s FTEDIT

program,
available on the NT Resource Kit, is a Microsoft supported tool designed to read and modify
the Disk key. Because of the binary structure of the Disk key, using FTEDIT is highly
recommended instead of editing the registry directly.
In the following sections, we describe the procedure for using FlashCopy and PPRC with
FTEDIT.
Preparation
On the target machine:
1.Back up the disk data using Disk Administrator and registry information using REGEDIT.
2.If the target disks have been previously used, delete all of the target disks in Disk
Administrator. Do not just unmount them, but delete all of the partitions on the target disks.
Commit the changes.
3.In the Control Panel, double-click Devices and make sure that Ftdisk is started, and is set
to start on boot. Ftdisk is the driver used by NT to identify and access fault tolerant drives
and other multi-volume logical drives, such as volume sets. If there are any such drives in
use on the system, Ftdisk will be started and set to start on boot. If it is not started, one
way to get it started is to create a multi-volume logical drive on a couple of spare disks.
This will require a reboot.
Attention: Incorrect use of FTEDIT could result in loss of access to software RAID arrays.
It is recommended that you use Disk Administrator to save your disk configuration before
using FTEDIT. In general, most errors made using FTEDIT are recoverable. For more
information on how to recover from FTEDIT errors, and on FTEDIT in general, see the
Microsoft Knowledge Base articles 131658 and 149927:
http://support.microsoft.com/?kbid=131658
http://support.microsoft.com/?kbid=149927

422
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
On the source machine:
Obtain the order in which the disks were added to the volume set. One way of doing this is to
use a freeware utility called
diskkey.exe
, available from:
http://www.sysinternals.com
This utility is not supported by IBM and is known to report disk numbering and other
information that is different than what Disk Administrator reports. However, the order in which
the disks are included in the volume set will be correct, and the correct ordering of the disks is
the information required to create a duplicate volume set on the target server.
Map the disks on the source machine to the disks on the target machine. For example,
determine that Disk6 on the source is FlashCopied to Disk9 on the target. One way to do this
is to look at the tasks on the ESS. If the FlashCopy task is set up to FlashCopy from volume
10017608 to volume 10117608, for example, then use
rsList2105s
from the Copy Services
CLI or the
datapath
command from SDD to show that 10017608 is Disk6 on the source
machine and 10117608 is Disk9 on the target machine. Map all of the disks in the volume set
to their counterpart on the target.
Performing the PPRC/FlashCopy
On the target machine:
1.Run the FlashCopy establish or PPRC terminate task(s).
2.Start Disk Administrator. If it asks you to write a signature on any of the disks, click No
(except in the special cases described below in the note). Once Disk Administrator is up,
commit the changes (this is very important), and close Disk Administrator.
3.Start FTEDIT by choosing Start

Resource Kit 4.0

Disk Tools

Fault Tolerance
Editor. Read the warning and click OK. There are two panes in the FTEDIT window. In
the left-hand pane is a list of the disks in the system. In the right-hand pane is the list of
partitions on that disk. You must add the disks to the volume set in the right order. Use the
results of diskkey.exe to determine the order in which the disks were added on the source
volume set.
Note: Disk Administrator will ask to write a signature when the FlashCopy is performed to
the same machine, because it detects a duplicate disk signature (the source and target
volumes have the same disk signature) and needs to write a new one. It is safe to do this,
but be sure that you are writing the signature to the FlashCopy target disk. If a signature is
written to the wrong disk, it may cause data corruption.
When FlashCopying to a different machine, usually the disk signature on the target
machine's disks are different than the FlashCopy source disks’ signature, so Disk
Administrator does not need to write a new signature to the target disks to use it. It is
unlikely, but possible, that by coincidence the disk signature of one of the source disks is
the same as one of the disks on the target machine. In this case, you will have to write a
signature on the target disk before using it. Again, it is safe to do this, but be sure that you
are writing the signature to the right disk.
Attention: Writing a new disk signature to the source disk may cause severe problems.
For example, a cluster disk will not become online after a change of its signature.

Chapter 8. Open systems specifics
423
4.Click Make FT set in the lower left-hand corner. When it asks you what kind of set you
would like, choose Volume set and click OK.
5.Click the first target disk in the left-hand pane. The list of partitions on that disk should
appear in the right-hand pane. Choose the partition that contains the volume set on that
disk (usually Partition 1).
6.Double-click Partition 1 in the right-hand pane. This will add this disk/partition to the
volume set in order.
7.Repeat steps 5 and 6 for the rest of the disks. If you make a mistake, you can cancel and
start from scratch. The disks
must
be added in the right order.
8.When all of the disks have been added, choose Save FT set at the bottom.
9.Click Save Changes to System in the Edit menu. Close FTEDIT.
10.Reboot.
11.When NT restarts, start Disk Administrator. The target disks should be yellow now,
indicating that they are in a volume set. Assign a drive letter and commit the changes. If
the drives are not usable at this point, then the disks were probably added in the wrong
order.
As long as the disk configuration does not change on the source or target, FlashCopy should
work as expected. If the disk configuration is changed in any way, such as adding an
additional disk to the volume set or rearranging the disks, then you will have to perform this
procedure again.
8.3 Windows 2000 and Copy Services
Windows 2000 handles its disks differently than does Windows NT. Windows 2000
incorporates a stripped-down version of the VERITAS Volume Manager, called the Logical
Disk Manager (LDM).
With the LDM, you are able to create logical partitions, perform disk mounts, and create
dynamic volumes. There are five types of dynamic volumes: simple, spanned, mirrored,
striped, and RAID-5.
On Windows NT, the information relating to the disks was stored in the Windows NT registry.
With Windows 2000, this information is stored on the disk drive itself in a partition called the
LDM database, which is kept on the last few tracks of the disk. Each volume has its own
128-bit Globally Unique Identifier (GUID) and belongs to a disk group. This is similar to the
concept of Physical Volume Identifier (PVID) and Volume Group in AIX. As the LDM is stored
on the physical drive itself, with Windows 2000 it is possible to move disk drives between
different computers.
Attention: If there are active PPRC target volumes on the target, then the disk
numbering used in FTEDIT may differ from the disk numbering used in Disk
Administrator. The PPRC target volumes are not seen by FTEDIT, and so are not
included in the disk numbering scheme. Adjust your disk choices accordingly.

424
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Copy Services limitations with Windows 2000
Having the drive information stored on the disk itself imposes some limitations when using
Copy Services functionality on a Windows 2000 system:

The source and target volumes must be of the same physical size. Normally the target
volume can be bigger than the source volume; with Windows 2000, this is not the case, for
two reasons:
a.The LDM database holds information relating to the size of the volume. As this is
copied from the source to the target, if the target volume is a different size from the
source, then the database information will be incorrect, and the host system will return
an exception.
b.The LDM database is stored at the end of the volume. The copy process is a
track-by-track copy; unless the target is an identical size to the source, the database
will not be at the end of the target volume.

It is not possible to have the source and target FlashCopy volume on the same
Windows 2000 System when they were created as Windows 2000 dynamic volumes. The
reason is that each dynamic volume has to have its own 128-bit GUID. As its name
implies, the GUID must be unique on one system. When you perform FlashCopy, the
GUID gets copied as well, so this means that if you tried to mount the source and target
volume on the same host system, you would have two volumes with exactly the same
GUID. This is not allowed, and you will not be able to mount the target volume.
8.3.1 Copy Services with Windows 2000 volumes
Basic disks are the same as the NT disks with the same restrictions. Dynamic disks are
supported for both PPRC and FlashCopy and the primary/source and secondary/target
volumes must be attached to different servers. We also support the use of Spanned Volumes
with PPRC and FlashCopy on Dynamic Disks with the same attachment restriction.
Mounting a Copy Services target volume
In order to see target volumes on a second Windows 2000 host, you have to do the following:
1.Perform the PPRC/FlashCopy function onto the target volume. Ensure that when using
PPRC that the primary and secondary volumes were in duplex mode, and write I/O was
ceased prior to terminating the copy pair relationship.
2.Reboot the host machine on which you wish to mount the Copy Services target volume.
3.Right-click Open Computer Management, and then click Disk Management.
4.Find the disk that is associated with your volume. There are two “panes” for each disk; the
left one should read Dynamic and Foreign. It is likely that no drive letter will be
associated with that volume.
5.Right-click that pane, and select Import Foreign Disks. Select OK, then OK again. The
volume now has a drive letter assigned to it, and is of Simple Layout and Dynamic Type.
You can read/write to that volume.
Note: When using Windows dynamic disks, remember that to read FlashCopy targets if
the FlashCopy pair has been rescanned or if the server reading targets has been rebooted,
FlashCopy targets will appear as "foreign" disks to that server. Manual intervention will be
required to re-import those disks and restore operation.

Chapter 8. Open systems specifics
425
When performing subsequent PPRC/FlashCopies to the target volume, it is not necessary to
perform a reboot, because the target volume is still known to the target system. However, in
order to detect any changes to the contents of the target volume, you should remove the drive
letter from the target volume before doing the FlashCopy. Then, after carrying out the
FlashCopy, you restore the drive letter in order for the host it is mounted on to be able to
read/write to it.
There is a Windows utility, DiskPart, that enables you to script these operations so that
FlashCopy can be carried out as part of an automated backup procedure. DiskPart can be
found at the Microsoft download site
http://www.microsoft.com/downloads
with a search on
the key word DiskPart. A description of DiskPart commands can be found at the Web site:
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/diskpart.
mspx
Extending simple volumes
The Copy Services source may initially be a single
simple
volume. However, as requirements
change on the application server, the logical volume may be extended over two or more
volumes. However, you should not independently extend the target volumes, but let
Windows 2000 detect the correct sequence of the extended volumes during the import
process. For this reason, the target volumes should be reverted back to
basic
disks prior to
the initial FlashCopy after the source has been extended. The target server should also be
rebooted for disk manager to pick up the new volumes.
After the reboot, the volumes will be recognized as
foreign
disks, and you can proceed to
import them. Reboot of the target system on subsequent FlashCopy is not necessary until the
source volume has been further extended. When performing subsequent PPRC/FlashCopy to
the target volume, it is not necessary to perform a reboot because the target volume is still
known to the target system. However, in order to detect any changes to the contents of the
target volume, you should remove the drive letter from the target volume before doing the
FlashCopy. Then, after carrying out the FlashCopy, you restore the drive letter in order for the
host it is mounted on to be able to read/write to it.
There is a Windows utility, DiskPart, that enables you to script these operations so that
FlashCopy can be carried out as part of an automated backup procedure. DiskPart can be
found at the Microsoft download site
http://www.microsoft.com/downloads
with a search on
the key word DiskPart.
The DiskPart tool also provides a way to extend an existing partition into free space at the
end of the same logical drive. A description of this procedure can be found in the Microsoft
Knowledge Base, article 304736:
http://support.microsoft.com/?kbid=304736
Enlarging Extended/Spanned volumes
When you have extended
or
spanned

disks, the logical drive may in time grow to include
more of the initial volume (extended disk) or include additional volumes. When this occurs, it
is necessary, as before, to remove the target volume group information and revert the target
volumes back to basic disks. On the initial FlashCopy, it is necessary to reboot the target
Tip: Disable the Fast-indexing option on the source disk; otherwise, operations to that
volume get cached to speed up disk access. However, this means that data is not flushed
from memory and the target disk may have copies of files/folders that were deleted from
the source system.

426
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
server to configure the additional disks, and then import all the foreign disks that are part of
the volume group.
When performing subsequent PPRC/FlashCopy to the target volume, it is not necessary to
perform a reboot, because the target volume is still known to the target system. However, in
order to detect any changes to the contents of the target volume, you should remove the drive
letter from the target volume before doing the FlashCopy. Then, after carrying out the
FlashCopy, you restore the drive letter in order for the host it is mounted on to be able to
read/write to it.
Again, we refer to the Windows 2000 utility DiskPart, which enables you to script these
operations so that FlashCopy can be carried out as part of an automated backup procedure.
DiskPart can be found at the Microsoft download site
http://www.microsoft.com/downloads
;
search for the key word “DiskPart”.
8.3.2 PPRC and Windows 2000 spanned volumes
Here is the procedure we followed when carrying out the PPRC of a Windows 2000 spanned
volume set from server A to server B:
1.On the source server A, we created a Windows spanned volume set of multiple dynamic
disks.
2.We rebooted the target server B and we imported multiple target disks and wrote a disk
signature on each as basic disks.
3.We established PPRC between the source and target volumes.
4.Once the source and target volumes were synchronized, we terminated PPRC.
5.We rebooted the target host B.
6.We started Disk Manager; the PPRC target volumes were seen as Foreign Dynamic
Disks
.
7.The disks were imported into the target host and were seen as a spanned volume.
In order to demonstrate failback to the original setup, we carried out the following tasks:
1.We removed the original paths and re-established them in the reverse direction from B
to A.
2.We removed the spanned volume drive letter from the original source (the spanned
volume on server A).
3.We established PPRC from B to A and wrote some data onto the spanned volume.
4.PPRC was terminated.
5.We restored the drive letter to the spanned volume on server A.
6.The contents of the spanned volume could now be read from server A.
There is a Windows utility, DiskPart, that enables you to script these operations so that
FlashCopy can be carried out as part of an automated backup procedure. DiskPart can be
found at the Microsoft download site
http://www.microsoft.com/downloads
; search for the
key word “DiskPart”.
8.3.3 ESSSync utility: Available by RPQ
The
ESSSync
utility is available from IBM by RPQ (request for price quotation) only. This utility
flushes Windows buffers and enable you to access FlashCopy target volumes without the
need to unassign and then reassign drives letters. Here is the procedure for using it:

Chapter 8. Open systems specifics
427
1.Run
ESSSync
against the source and target volumes to flush all buffers (both volumes are
mounted on their respective hosts).
2.Perform the FlashCopy.
3.Run
ESSSync
against the target volume.
4.Access the data on the target volumes.
ESSSync
is run from the command line and can be incorporated into your backup scripts.
8.3.4 Relating ESS volumes to Windows drive letters
When logical volumes are created in the ESS, they are assigned a serial number. These
numbers are shown when you look at the volumes using the ESS Copy Services GUI. This
can be seen in Figure 8-9.
Figure 8-9 ESS logical volume serial numbers
Disk administrator will see the drives as shown in Figure 8-10 on page 428, that is, as Disk 1,
drive letter F and Disk 2, drive letter G.

428
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-10 View of disks as seen by Disk Administrator
If you have the CLI installed and you run the
rsList2105s
command, then you get the output
shown in Figure 8-11. This shows the relationship between Disk 1 and Disk 2 and the ESS
serial numbers. Note that Disk 3 and Disk 4 are the views of Disk 1 and Disk 2 seen via the
second Fibre Channel path. These views of the disk are not seen by applications.
Figure 8-11 Output from rsList2105s
Also, if you have the CLI installed on the host that is accessing the volumes, then you can run
the
rsPrimeServer
command. This will then show the Windows volume letters that have been
assigned to the ESS volumes when you view the Volumes panel in the Copy Services GUI
(see Figure 8-12 on page 429).

Chapter 8. Open systems specifics
429
Figure 8-12 View after rsPrimeServer has been run
The IBM SDD software can be used to carry out query commands to find the condition of
multipath devices attached to a Windows 2000 server. The two main commands used are

datapath query adapter
and
datapath query device
. This is shown in Figure 8-13.
Figure 8-13 Datapath query device command output
The
datapath query adapter
command will return a list of adapters, as illustrated in
Example 8-8 on page 430. The output should include information about all the installed
adapters. In this example, one SCSI adapter is installed on the Windows 2000 host server.

430
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Example 8-8 The datapath query adapter command
+--------------------------------------------------------------------------------+
|Active Adapters :1 |
| |
|Adpt# Adapter Name State Mode Select Errors Paths Active |
| 0 Scsi Port1 Bus0 NORMAL ACTIVE 4057 0 8 8 |
+--------------------------------------------------------------------------------+
8.4 SUN Solaris and Copy Services
In the following section, we describe the actions that should be taken to perform Copy
Services functions and mount a target volume on a SUN Solaris server.
Making a Copy Services target volume available to the same server or to another server is
possible.
You can use the Copy Services CLI for automation and create scripts to automate your
procedures. We recommend that you predefine the tasks to be run and test them thoroughly.
Also, prepare your target mount point.
8.4.1 Copy Services without a volume manager
In this section, we describe how to access Copy Services volumes under SUN Solaris without
volume manager software. Native commands are used to show how it is possible to access
the target volume after the Copy Services function has completed.
The shell script to be run before the application that will use FlashCopy target should include
the operations shown in Example 8-9.
Example 8-9 Backup preparation tasks
#quiesce an application
# insert the quiescing script here
#unmounting the source
umount /source
#start FlashCopy task
rsExecuteTask.sh -s CopyServicesServer EstablishTaskName
#check if FlashCopy task is established
rsQuery.sh -f VolumeList -s CopyServicesServer
#if yes, you can mount the source back
mount /source
#and resume the application
# insert the resuming script here
#check the target for consistency
fsck -y /dev/rdsk/cXtYdZsN
#if OK mount it
mount /dev/dsk/cXtYdZsN /target
The shell script to be run after backup is shown in Example 8-10.
Example 8-10 Post backup tasks
#unmount the target
umount /target
#terminate the FlashCopy pair if Do not perform background copy was used
rsExecuteTask.sh -s CopyServicesServer WithdrawTaskName

Chapter 8. Open systems specifics
431
The foregoing steps can be performed for PPRC as well as FlashCopy by substituting the
FlashCopy task with a PPRC terminate copy pair task on duplex volumes. Once the operation
on the secondary volume has completed, then, rather than execute the FlashCopy withdraw,
you can perform a PPRC establish copy pair task. This operation can be performed again
once the volumes have become duplex again.
In the following example, we describe a method of accessing the FlashCopy target on a single
SUN Solaris system. The FlashCopy source is active on that server at the same time. This
example can also be applied to a PPRC secondary volume on a target SUN Solaris server.
In our example, there is a file system named /source on the source volume c1t6d0s2 of the
FlashCopy pair. The target of the FlashCopy will be c1t6d1s2.
You can display all available ESS LUNs using the
rsList2105s.sh
command of the Copy
Services CLI. This can be seen in Figure 8-14.
Figure 8-14 rsList2105s.sh on SUN Solaris
Make sure that the PPRC/FlashCopy source is in a consistent state for the short period
during the establishment of the FlashCopy pair or prior to terminating the PPRC relationship.
Use the following procedure to access the target volume:
1.Bring down applications that access the PPRC/FlashCopy primary/source. Unmount the
related file systems for the short period of PPRC/FlashCopy termination/establishment:
umount /source
2.Using the Copy Services Web Interface or the Command Line Interface, terminate the
PPRC relationship or establish FlashCopy.
3.Once the PPRC/FlashCopy pair is terminated/established, mount all file systems and
restart the applications. This could be done even if data is still copied from the source to
the target in the background when using FlashCopy. Run:
mount /dev/dsk/c1t6d0s2 /source
4.Check the consistency of the file system on the target volume:
fsck -y /dev/rdsk/c1t6d1s2
5.Create a mount point for the target file system and mount the file system:
mkdir /target
mount /dev/dsk/c1t6d1s2 /target
Now the data on the target could be accessed. The
mount -v
output in Figure 8-15 on
page 432 shows that the source and target are active at the same time in the case of a
FlashCopy between source and target volume on the same system.
./rsList2105s.sh
disk name 2105 serial number
--------- ------------------
c1t6d0 500FCA24
c1t6d1 501FCA24
c1t6d2 502FCA24
c1t6d3 503FCA24
c1t6d4 504FCA24
c1t6d5 505FCA24

432
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-15 Output from mount -v on Solaris
8.4.2 Copy Services with a VERITAS Volume Manager
In the following section, we describe how to perform FlashCopy and PPRC on SUN Solaris
systems with VERITAS Volume Manager (VxVM) support.
FlashCopy with VERITAS Volume Manager
Here we explain how to simultaneously mount ESS FlashCopy source and target volumes to
the same host without exporting the source volumes when using VERITAS Volume Manager.
It is assumed that the sources are constantly mounted to the SUN host, the FlashCopy is
performed, and the goal is to mount the copy without unmounting the source or rebooting.
After the target volumes have been assigned, it is necessary to reboot the SUN server using

reboot -- -r
or, if a reboot is not immediately possible, then issue
drvconfig
,
disks
, and
then
devlinks
. However, a reboot is recommended for guaranteed results.
It is also assumed that the appropriate actions in order to use the target volumes with the host
have already taken place (that is,
devfsadm
,
vxdctl enable
...).
The following procedure refers to these names:

mydg: As the name of the diskgroup that is being created.

da_name: As the disk name shown under the DISK column in the
vxdisk list
output.

last_daname: As the disk is known to VxVM as, as shown under DEVICE column in the
vxdisk list
output. This is the output of vxdisk list on SUN Solaris.
Use the following procedure to mount the targets to the same host:
1.Determine which disks have a copy of the disk group configuration in their private region.
The following command will list the
log disk
disks:
# vxdg list <disk group>
2.Determine the location of the private region (tag 15) on the disks (normally partition 3):
# prtvtoc /dev/rdsk/c#t#d#s2
or use the following command to get the partition number for the private region:
# vxdisk list c#t#d#s2 | grep priv
3.Dump the private region:
# /usr/lib/vxvm/diag.d/vxprivutil dumpconfig /dev/rdsk/c#t#d#s3 > dg.dump
4.Create a script to initialize the disk group:
# cat dg.dump | vxprint -D - -d -F "vxdg -g <mydg> adddisk %name=%last_da_name" > dg.sh
5.Edit the file dg.sh and change the first line to:
# vxdg init <mydg> <daname>=<last_daname>
6.Make the file dg.sh executable:
# chmod 755 dh.sh
/dev/dsk/c1t6d0s2 on /source type ufs read/write/setuid/largefiles...
/dev/dsk/c1t6d1s2 on /target type ufs read/write/setuid/largefiles...

Chapter 8. Open systems specifics
433
7.Create a file that can be used to rebuild the VM config:
# cat dg.dump | vxprint -D - -hvpsm > dg.maker
8.Initialize the disk group by executing dg.sh:
# ./dh.sh
9.If this results in the error
Disk is already in use by another system
, then the private
region on each disk that is to be added to the disk group will need to be initialized. This
can be done with the following command:
# vxdisksetup -i <da_name>
10.Rebuild the VM configuration:
# vxmake -g <mydg> -d dg.maker
11.Start the volumes:
# vxvol -g <mydg> start <volume>
PPRC with VERITAS Volume Manager
In the previous section, we described how to perform a FlashCopy and mount the source and
target file system on the same server. Here we describe the steps necessary to mount a
PPRC secondary volume onto a server that does not have sight of the primary volume.
It assumes that the PPRC copy pair has been terminated prior to carrying out the procedure.
After the secondary volumes have been assigned, it is necessary to reboot the SUN server
using
reboot -- -r
or, if a reboot is not immediately possible, then issue
drvconfig
,
disks

and then
devlinks
. However, a reboot is recommended for guaranteed results.
Use the following procedure to mount the secondary volumes to another host:
1.Scan devices in the operating system device tree:
vxdisk scandisks
2.List all known disk groups on the system:
vxdisk -o alldgs list
3.Import the PPRC disk group information:
vxdg -C import <disk_group_name>
4.Check the status of volumes in all disk groups:
vxprint -Ath
5.Bring the disk group online:
vxvol -g <disk_group_name> startall
or
vxrecover -g <disk_group_name> -sb
6.Perform a consistency check on the file systems in the disk group:
fsck -V vxfs /dev/vx/dsk/<disk_group_name>/<volume_name>
7.Mount the file system for use:
mount -V vxfs /dev/vx/dsk/<disk_group_name>/<volume_name> /<mount_point>

434
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Once you have finished with the PPRC secondary volume, it is recommended that you
perform the following tasks:
1.Unmount the file systems in the disk group:
umount /<mount_point>
2.Offline the volumes in the disk group:
vxvol -g <disk_group_name> stopall
3.Export disk group information from the system:
vxdg deport <disk_group_name>
8.5 HP-UX and Copy Services
The following section describes how it is possible to access a source and target Copy
Services volume on the same HP server.
8.5.1 HP-UX with FlashCopy
The following procedure must be followed to permit access to the FlashCopy source and
destination simultaneously on an HP-UX host. It could be used to make an additional copy of
a development database for testing or to permit concurrent development, to create a
database copy for data mining that will be accessed from the same server as the OLTP data,
or to create a Point-In-Time copy of a database for archiving to tape from the same server.
This procedure must be repeated each time you perform a FlashCopy and want to use the
target physical volume on the same host where the FlashCopy source volumes are present in
the Logical Volume Manager configuration.
You can use the Copy Services CLI for automation and create scripts to automate your
procedures. If you are preparing scripts, you must also prepare your FlashCopy tasks and
test them.
Target preparation
In order to prepare the target system, carry out the following steps:
1.Vary off the source volume groups:
vgchange -a n /dev/<source_vg_name>
2.If you did not use the default Logical Volume Names (lvo
ln
n) when they were created,
create a map file from your source volume group using the
vgexport
command:
vgexport -m <map file name> -p /dev/<source_vg_name>
3.If the target volume group exists, remove it using the
vgexport
command. The target
volumes cannot be members of a volume group when the
vgimport
command is run:
vgexport -m /dev/null /dev/<target_vg_name>
Tip: If you FlashCopy or PPRC only one half of a RAID-1 mirror, it will be necessary to
force the import of the disk group because not all of the disks are available. Therefore, it is
necessary to issue the following command:
vxdg -f import <disk_group>
However, be aware that this may cause disk group inconsistencies.
Note: This map file needs to be “ftp’d” to the target host.

Chapter 8. Open systems specifics
435
4.Shut down or quiesce any applications that are accessing the FlashCopy source.
FlashCopy execution
To execute the procedure, you must carry out the following steps:
1.Unmount all file systems in the source volume group.
2.Perform the FlashCopy.
3.Ensure that the FlashCopy task has completed using the ESS Specialist Web Interface or
the Command Line Interface command
rsQuery.sh
:
./rsQuery.sh -u <CLI_User> -p <CLI_Password> -f disklist -s\
<primary_copy_services_server>
4.Mount all the file systems in the source volume group.
5.When the FlashCopy is finished, change the Volume Group ID on each ESS Volume in the
FlashCopy target. The volume ID for each volume in the FlashCopy target volume group
must be modified on the same command line. Failure to do this will result in a mismatch of
Volume Group IDs within the volume group. The only way to resolve this issue is to
perform the FlashCopy again and reassign the Volume Group IDs using the same
command line:
vgchgid -f </dev/rdsk/c#t#d#_1>...</dev/rdsk/c#t#d#_n>
6.Create the Volume Group for the FlashCopy target:
mkdir /dev/<target_vg_name>
mknod /dev/<target_vg_name>/group c <lvm_major_no> <next_available_minor_no>
Use the
lsdev -C lvm
command to determine what the major device number should be for
Logical Volume Manager objects. To determine the next available minor number, examine
the minor number of the group file in each volume group directory using the
ls -l
command.
7.Import the FlashCopy target volume(s) into the newly created volume group using the
vgimport
command:
vgimport -m <map file name> -v /dev/<target_vg_name>
</dev/dsk/c#t#d#_1>...</dev/dsk/c#t#d#_n>
8.Activate the new volume group:
vgchange -a y /dev/<target_vg_name>
9.Perform a full file system check on the logical volumes in the target volume group. This is
necessary in order to apply any changes in the JFS intent log to the file system and mark
the file system as
clean
.
fsck -F vxfs -o full -y /dev/<target_vg_name>/<logical volume name>
10.If the logical volume contains a VxFS file system, mount the target logical volume(s) on the
server:
mount -F vxfs /dev/<target_vg_name>/<logical volume name><mount point>
The example in Figure 8-16 on page 438 shows how to import a FlashCopy target on the
same system as the source but with the default logical volume naming scheme.
Once access to the FlashCopy target volume is no longer required, unmount the file systems
and vary off the volume group:
vgchange -a n /dev/<target_vg_name>
Note: This step is not needed if another host is used to access the target devices.

436
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
If no changes are made to the source volume group prior to the subsequent FlashCopy, then
all that is needed is to vary on the volume group and perform a full file system consistency
check, as shown in steps 8 to 10.
8.5.2 HP-UX with PPRC
When using PPRC with HP-UX, it is similar to using FlashCopy, apart from the fact that the
volume group should be unique to the target server, so there should be no need to perform
the
vgchgid
command to change the physical volume to volume group association. Here is
the procedure to bring secondary volumes online to PPRC target HP-UX hosts:
1.Quiesce the source HP-UX application to cease any updates to the primary volumes.
2.If you are using PPRC-XD, allow the copy pair volumes to go into duplex state using the
catch-up operation or by leaving the volumes to become synchronized.
3.Terminate the PPRC copy pair relationship using the Copy Services Web User Interface
or by using the CLI.
4.Rescan for hardware configuration changes using the
ioscan -fnC disk
command. Check
that the disks are
CLAIMED
using
ioscan -funC disk
. The reason for doing this is that
the volume group may have been extended to include more physical volumes.
5.Create the Volume Group for the PPRC secondary. Use the
lsdev -C lvm
command to
determine what the major device number should be for Logical Volume Manager objects.
To determine the next available minor number, examine the minor number of the group file
in each volume group directory using the
ls -l
command.
6.Import the PPRC secondary volume(s) into the newly created volume group using the
vgimport
command.
7.Activate the new volume group.
8.Perform a full file system check on the logical volumes in the target volume group. This is
necessary in order to apply any changes in the JFS intent log to the file system and mark
the file system as
clean
.
9.If the logical volume contains a VxFS file system, mount the target logical volume(s) on the
server.
If changes are made to the source volume group, they should be reflected in the /etc/lvmtab
of the target server. Therefore, it is recommended that periodic updates be made to make the
lvmtab on both source and target machines consistent. As with the AIX
importvg
, there are
two alternatives:

Using the Permit read from secondary option:
a.If you are using PPRC-XD, issue
go-to-sync
to allow the volumes to go to duplex state.
b.Once the volumes are in the duplex state, suspend the primary volume so that no
updates are reflected on the secondary volumes.
c.Export the source volume group information into a map file.
d.Export the old volume group definitions from the target host.
e.Run an
ioscan
to identify any new volumes that have been assigned to the target hosts
due to expansion of the source volume group.
f.Import the target volume group definition using the map file generated from the source
host.
g.Reestablish the PPRC relationship, only copying cylinders out-of-sync.

Chapter 8. Open systems specifics
437

Using NOCOPY PPRC establish:
a.Quiesce all write I/O to the primary volumes, and unmount the source file systems.
b.If you are using PPRC-XD, issue the
go-to-sync
command to all the secondary
volumes to catch up.
c.Once the volumes are in duplex state, terminate the PPRC relationship.
d.Export the source volume group information into a map file.
e.Export the old volume group definitions from the target host.
f.Run an
ioscan
to identify any new volumes that have been assigned to the target hosts
due to expansion of the source volume group.
g.Import the target volume group definition using the map file generated from the source
host.
h.Establish the PPRC relationship with the NOCOPY option, so that only the primary and
secondary have a copy pair relationship without updates.
i.Immediately suspend the primary volumes.
j.Mount the file systems and start the application at source.
k.Some time later, reestablish PPRC, only copying the out-of-sync cylinders.
Refer to the code sample in Figure 8-16 on page 438, which shows the import of a FlashCopy
target.

438
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-16 Import of a FlashCopy target
8.6 OpenVMS with Copy Services
In the following section, we discuss the necessary tasks to perform when bringing a Copy
Services target volume online on an HP OpenVMS server.
8.6.1 OpenVMS with FlashCopy
Under OpenVMS, volumes have labels associated to them. The volume label acts as a
unique volume identification cluster-wide. When you perform a FlashCopy, all the disk
structures get copied from the source, so when the target is brought online, it will have the
same information as the source and therefore the same volume label. This causes a volume
# umount /test
# ./rsExecuteTask.sh -u storwatch -p specialist -s sls6c1 HPFlash
# ./rsQuery.sh -u storwatch -p specialist -f disklist -s sls6c1
16-Oct-02 3:42:16 PM rsClientImpl: Successfully connected to Name Server, port = 1703
16-Oct-02 3:42:17 PM rsClientImpl: Server identity = IBM2105 Copy Services/1.1.0
16-Oct-02 3:42:18 PM rsClientImpl: Received rsVSServer reference successfully
16-Oct-02 3:42:19 PM rsClientImpl: rsClientImpl registered successfully
************************Volume Information************************
Volume 20014744 found on 14744:12 as volume number 000
State=simplex, status=not_suspended, FlashCopy_state=source, Size=20.0_GB
******************************************************************
************************Volume Information************************
Volume 20114744 found on 14744:12 as volume number 001
State=simplex, status=not_suspended, FlashCopy_state=target, Size=20.0_GB
******************************************************************
# mount /test
# mkdir /dev/vgibm
# mknod /dev/vgibm/group c 64 0x040000
# vgimport -v /dev/vgibm /dev/dsk/c0t6d2
Beginning the import process on Volume Group "/dev/vgibm".
Logical volume "/dev/vgibm/lvol1" has been successfully created
with lv number 1.
Volume group "/dev/vgibm" has been successfully created.
Warning: A backup of this volume group may not exist on this machine.
Please remember to take a backup using the vgcfgbackup command after activating
the volume group.
# vgchange -a y /dev/vgibm
Activated volume group
Volume group "/dev/vgibm" has been successfully changed.
# vgcfgbackup /dev/vgibm
Volume Group configuration for /dev/vgibm has been saved in /etc/lvmconf/vgibm.c
onf
# fsck -F vxfs -p -y -o full /dev/vgibm/lvol1
/dev/vgibm/lvol1:log replay in progress
/dev/vgibm/lvol1:pass0 - checking structural files
/dev/vgibm/lvol1:pass1 - checking inode sanity and blocks
/dev/vgibm/lvol1:pass2 - checking directory linkage
/dev/vgibm/lvol1:pass3 - checking reference counts
/dev/vgibm/lvol1:pass4 - checking resource maps
/dev/vgibm/lvol1:OK to clear log? (ynq)y
/dev/vgibm/lvol1:set state to CLEAN? (ynq)y
# mount /dev/vgibm/lvol1 /ibm

Chapter 8. Open systems specifics
439
label conflict when the FlashCopy target should be mounted at the same host or in a host
belonging to the same OpenVMS cluster.
There are two ways of overcoming this problem:

Dismount the source volume and mount the target volume, therefore not having two
volumes with the same volume label mounted at the same time.

Mount the target volume for private access only by a single process.
If the target volume is needed only for some specific operation (like creating a backup),
then this can be done in that process. If the target is needed for global access, then you
change the target volume label, dismount it, and mount it again for global access.
The next procedure shows how you can perform a FlashCopy on a source volume and the
steps required for changing the volume label on the target and mounting it for global access:
1.Once the target volumes have been assigned using the ESS StoreWatch Specialist, it is
necessary to configure them to the system using the following command:
mcr sysman io autoconfigure/log
2.Identify whether the new volumes assigned to be the FlashCopy targets are visible to the
system, as shown in Figure 8-17, using the following command:
show devices dk
Figure 8-17 Output showing attached SCSI disks
The
show device
command shows you devices of a particular type. In this case, it is a disk
device with a SCSI (dk) attachment. If you wanted to see disk devices with Fibre Channel
attachment, then dg would be the device specifier (see Figure 8-18 on page 440).
The best way of relating the attached devices to volumes configured in the ESS is to use
the Copy Services CLI command
rsList2105s
. However, if you are using SCSI attached,
you can translate the physical device name, for example, ES40$DKB601:
– The letters before the $ sign (ES40) are the OpenVMS host name.
– The first two letters after the $ sign (DK) specify the device and connection type as
SCSI disk.
– The third letter (B) denotes the SCSI adapter placement.
Note: Care must be taken on the sizing and placement of volumes in a SCSI attached
environment, because OpenVMS supports eight LUN-IDs per SCSI target, giving a
maximum of 120 volumes per adapter. In a Fibre Channel attached environment, the User
Defined ID (UDID) must be unique throughout the OpenVMS cluster.
Note: You need the OPER privilege to run the SYSMAN utility. Additionally, you must
have CMKRNL and SYSLCK privileges to use the
mcr sysman io autoconfigure

command.

440
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
– The final three digits indicate the SCSI address, where the first digit is the SCSI target
ID and the last two digits are the LUN ID. The volumes used in this example were
ES60$DKB600 (source) and ES40$DKB601 (target).
The device name for disks with Fibre Channel attachment has the structure
$1$DGAnnnnn (see Figure 8-18):
– The $1 is a so-called storage allocation classes, replacing the OpenVMS host name.
– The first two letters after the $ sign (DG) specify the device and connection type as a
Fibre Channel disk.
– The third letter is always A for any FC disk.
– The final digits (up to five) are the
User-Defined ID
(UDID), which is stored as a label
in the ESS. (This label is an ESS-internal identification string, different from the
operating system’s volume label.)
Figure 8-18 Output showing attached FC disks
An ESS volume label that represents a number in the range 0–32767 is presented as
UDID to OpenVMS hosts. Figure 8-19 on page 441 shows the volume panel with the ESS
volume labels 2001, 2002, and 2003.
VMS1$ show device dg
Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
$1$DGA2001: (VMS1) Mounted 0 QUORUM 1952886 1 1
$1$DGA2002: (VMS1) Mounted 0 V73_COMMON 1067115 357 1
$1$DGA2003: (VMS1) Online 0
$1$DGA2004: (VMS1) Online 0
$1$DGA2005: (VMS1) Online 0
VMS1$

Chapter 8. Open systems specifics
441
Figure 8-19 OpenVMS volumes with UDID labels
3.Perform the FlashCopy operation using the Copy Services Web User Interface
(Figure 8-20 on page 442) or via the CLI. Please note the quotation marks around user
name, password, and task name. These items are case-sensitive in the ESS, but the
OpenVMS command interpreter would convert them to uppercase without quotation
marks. (Currently, there is only the Copy Services CLI available for OpenVMS.) Run:
rsExecuteTask -u ”storwatch” -p ”specialist” -s sls6c1 ”OpenVMSFlash”

442
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-20 Established source and target
Now that the FlashCopy volumes are established, it is possible to access the target volume in
read-only mode. This is shown in Figure 8-21. It is not strictly necessary to mount the target
volume as read-only the first time, but we recommend that as “good practice” to look at the
volume before doing anything. The important thing is that this mount operation was a private
one (not system-wide or cluster-wide), making the volume accessible from only that process
that issued the
mount
command.
Figure 8-21 Checking consistency of FlashCopy target volume
Although we mount the FlashCopy target volume, we must refer to it with the same OpenVMS
volume label as assigned to the source volume. Figure 8-22 on page 443 shows that although
the FlashCopy target is mounted, it has a writelock initiated. The keyword alloc indicates that
the device is allocated for private access by a single process.
$ mount /nowrite dkb601: e600
%MOUNT-I-MOUNTED, E600 mounted on _ES40$DKB601:
$ dir dkb601:[000000]
Directory DKB601:[000000]
000000.DIR;1 BACKUP.SYS;1 BADBLK.SYS;1 BADLOG.SYS;1
BITMAP.SYS;1 CONTIN.SYS;1 CORIMG.SYS;1 FLASH_TEST.TXT;1
INDEXF.SYS;1 REPEAT_COPY_00000433_1.TMP;1
REPEAT_COPY_00000437_1.TMP;1 REPEAT_COPY_00000438_1.TMP;1
SECURITY.SYS;1 VOLSET.SYS;1
Total of 14 files.

Chapter 8. Open systems specifics
443
Figure 8-22 Status of FlashCopy mount
Now that the FlashCopy target contents have been verified, it is necessary to dismount the
target volume and remount it for write access, again for the use by a specific process, in order
to not conflict with the source that is mounted for global access.
We notice that the message,
%MOUNT-I-REBUILD, volume was improperly dismounted;
rebuild in progress
in Figure 8-23 comes from the nonzero mount counter on the target
volume: OpenVMS manages a mount counter in every volume's home block. Because this
number was greater than zero on the source volume when creating the copy, it is also greater
than zero on the target volume. Therefore, when mounting the target volume with write
access for the first time, OpenVMS suspects an incorrect dismount, so it checks and repairs
the volume. (This is suppressed when mounting with the /nowrite qualifier, because this check
and repair always requires write access.)
Figure 8-23 Mount FlashCopy target for write
4.The target still retains the volume label that was copied over from the FlashCopy source;
however, because the volume is allocated for exclusive use by the ID, there is no conflict.
Since the volume is mounted in exclusive read/write mode, we can change the volume
label to what we desire. This is shown in Figure 8-24 on page 444.
$ show device dk
Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
ES40$DKA0: Mounted 0 ES4073 14988114 1 1
ES40$DKA100: Mounted 0 ES40_73 13801842 312 1
ES40$DKA200: Online 0
ES40$DKB200: Mounted 43 E200 19144495 1 1
ES40$DKB207: Mounted 43 E207 19142802 1 1
ES40$DKB600: Mounted 43 E600 16341488 1 1
ES40$DKB601: Mounted alloc 43 E600 15221698 1 1
WriteLocked
$
$ dismount dk601:
$ mount/override=id dkb601:
%MOUNT-I-MOUNTED, E600 mounted on _ES40$DKB601:
%MOUNT-I-REBUILD, volume was improperly dismounted; rebuild in progress
$ show device dk
Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
ES40$DKA0: Mounted 0 ES4073 14988114 1 1
ES40$DKA100: Mounted 0 ES40_73 13801842 312 1
ES40$DKA200: Online 0
ES40$DKB200: Mounted 43 E200 19144495 1 1
ES40$DKB207: Mounted 43 E207 19142802 1 1
ES40$DKB600: Mounted 43 E600 16341488 1 1
ES40$DKB601: Mounted alloc 43 E600 15221698 1 1
$

444
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-24 Change FlashCopy target volume label
5.Since the target volume has now a unique label, it is possible to change the access to
system-wide or cluster-wide so that other processes can gain access to the data on the
volume (see Figure 8-25).
Figure 8-25 Providing system-wide access to FlashCopy target
Subsequent FlashCopies to the target volume will require the target to be dismounted,
whether it be the same or another host system.
8.6.2 OpenVMS and PPRC
When we assume that the PPRC secondary volume is mounted at a different host that is not
in the same OpenVMS cluster as the host with the primary volume, then there is no volume
label conflict. Therefore, there is no need to change the volume label prior to mounting it for
system use.
$ set volume/label=e601 dkb601:
$ show device dk
Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
ES40$DKA0: Mounted 0 ES4073 14988114 1 1
ES40$DKA100: Mounted 0 ES40_73 13801842 312 1
ES40$DKA200: Online 0
ES40$DKB200: Mounted 43 E200 19144495 1 1
ES40$DKB207: Mounted 43 E207 19142802 1 1
ES40$DKB600: Mounted 43 E600 16341488 1 1
ES40$DKB601: Mounted alloc 43 E601 15221698 1 1
$
$ dismount dk601:
$ mount/system dkb601: e601
%MOUNT-I-MOUNTED, E601 mounted on _ES40$DKB601:
$ show device dk
Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
ES40$DKA0: Mounted 0 ES4073 14988114 1 1
ES40$DKA100: Mounted 0 ES40_73 13801842 312 1
ES40$DKA200: Online 0
ES40$DKB200: Mounted 43 E200 19144495 1 1
ES40$DKB207: Mounted 43 E207 19142802 1 1
ES40$DKB600: Mounted 43 E600 16341488 1 1
ES40$DKB601: Mounted 43 E601 15221698 1 1
$
Note: You need the SYSPRV and SYSNAM privileges to mount a volume for system-wide
or cluster-wide access.

Chapter 8. Open systems specifics
445
The following procedural steps are almost exactly the same as those of the FlashCopy;
however, certain steps have been excluded due to the assumption that the primary and the
secondary volume will not be on the same server/cluster:
1.Quiesce write I/O to the PPRC primary volume.
2.Ensure that the PPRC copy pair is in a duplex state. If you are using PPRC-XD, perform
the catch-up operation (see Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on page 55).
3.Terminate the PPRC copy pair relationship, using the Copy Services Web Interface or the
CLI.
4.If the secondary volume is not recognized by the other host, use the
mcr sysman io
autoconfigure
command to configure the volume.
5.Verify that the secondary volume is recognized by the target system using the
show dev dk
(or
dg
for Fibre Channel)

command.
6.Mount the secondary volume for system-wide access using the
mount/system
command
(or for cluster-wide access using the
mount/cluster
command).
8.7 Tru64 and Copy Services
In this section, we describe how to perform Copy Services functions such as FlashCopy and
PPRC on an HP Tru64 system. We explain how to perform Copy Services functions on
volumes in a clustered environment and how to deal with
Persistent Reserves
placed on the
target volumes. We also discuss how to handle Advance Filesystem (AdvFS) file systems so
Copy Services target volumes containing backup data can be mounted under the same
domain.
8.7.1 Tru64 and FlashCopy
The following section describes the procedure to perform FlashCopy on a Tru64 5.x system
consisting of UNIX Filesystem (UFS) and Advance Filesystem (AdvFS) file systems. We
explain how to mount the source and target FlashCopy volumes containing UFS file systems
on a Tru64 5.x server.
In a Tru64 5.x clustered environment, ownership of shared SCSI devices is controlled by
placing Persistent Reserves. The symptom of this is that initiation of Copy Services tasks on
target volumes fail. Therefore, prior to initiating any Copy Services tasks on target volumes, it
is necessary to check for reserves placed on target volumes and deciding whether to release
them prior to instigating a copy function.
The procedure uses the essvol script shown in Figure 8-26 on page 446 to identify assigned
ESS volumes.
Important: Prior to re-establishing the PPRC copy pair, ensure that you
dismount
the
PPRC secondary volume from the target system. Do not mount a PPRC secondary
volume while it is established in a copy pair relationship, as this will cause a system crash.
This warning applies also if the Permit read from secondary option has been activated.
To prohibit mount operations (which might be performed even by non-privileged users), you
should set the secondary volume as considered unavailable using the command
set
device/noavailable
.

446
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-26 essvol script
UFS Filesystem on Tru64 and FlashCopy
The following procedure describes the steps necessary to perform a FlashCopy to a target
volume and mount it on the same server as the source volume.
1.Check whether the FlashCopy target is assigned to the Tru64 host (Figure 8-27):
./essvol
Figure 8-27 Output of essvol script
2.Use the serial_number field to identify the volumes being used from the host side. The last
eight digits denote the volume being used, and the breakdown for 1042-1380 is as follows:
– 104 denotes the volume 04 on LSS 11.
– 2-1380 denotes the ESS serial number of 21380.
3.Quiesce all write I/O and unmount file systems from source volumes.
4.Ensure that the target volume has no file systems mounted.
5.Check and clear the persistent reserve on target volume if in a clustered environment (see
8.7.3, “Managing Tru64 Persistent Reserve” on page 449).
6.Perform FlashCopy from dsk43 (ESS volume 104) to dsk47 (ESS volume 10a).
7.Remount the file systems in the FlashCopy source volume.
8.Once the FlashCopy has established, perform a consistency check on the target volume
file systems:
fsck /dev/disk/dsk47c
9.Mount the file systems on the target volume:
mount /dev/disk/dsk47c /ess
This procedure is the same if you are mounting the FlashCopy target onto a different Tru64
server from the FlashCopy target.
echo Extracting ESS volume information...
for ID in `hwmgr -view dev -cat disk | egrep -e 2105F20 -e 2105800 | awk '{ print $1}'`
do
echo; echo ESS vol, H/W ID $ID
hwmgr -get attrib -id $ID | awk '/phys_loc//dev_base//capacity//serial/'
done
<alpha1> essvol
ESS vol, H/W ID 176:
phys_location = bus-9-targ-0-lun-0
dev_base_name = dsk43
capacity = 3906304
serial_number = SCSI-WWID:01000010:6000-1fe1-4942-4d20-0000-0000-1042-1380

ESS vol, H/W ID 225:
phys_location = bus-9-targ-0-lun-7
dev_base_name = dsk47
capacity = 3906304
serial_number = SCSI-WWID:01000010:6000-1fe1-4942-4d20-0000-0000-10a2-1380

Chapter 8. Open systems specifics
447
AdvFS Filesystem on Tru64 and FlashCopy
The following procedure describes the steps necessary to mount a FlashCopy target AdvFS
volume to the same or another server in the same cluster. To mount the FlashCopy target,
it is necessary to manually create the AdvFS domain/fileset and link it to the FlashCopy target
volume.
For our example (shown in Figure 8-28), we use an AdvFS volume consisting of three ESS
disks. The FlashCopy source AdvFS domain is called
DOM1
with a fileset name of advol1
and consists of ESS disks: dsk58, dsk59, and dsk60. The FlashCopy target disks are dsk62,
dsk63, and dsk64, and will belong to AdvFS domain DOM2.
1.Ensure that the FlashCopy source and target volumes are identical in size.
2.Check the contents of AdvFS domain DOM1.
Figure 8-28 Contents of AdvFS domain DOM1
3.Manually create the AdvFS domain used for the FlashCopy target volumes (Figure 8-29).
Figure 8-29 Manual AdvFS domain creation
4.Unmount the file systems associated to the FlashCopy source volumes.
5.Check and clear the persistent reserve on the target volume if in a clustered environment
(see 8.7.3, “Managing Tru64 Persistent Reserve” on page 449)
6.Perform FlashCopy between the source (dsk58, dsk59 and dsk60) and target (dsk62,
dsk63 and dsk64) volumes.
7.Once the FlashCopy has been established, you can proceed to remount the file systems
on the FlashCopy source volumes or mount the file systems on the target volumes:
mount -t advfs DOM2#advol1 /ess/advol2
File systems on both source and target cannot be mounted simultaneously on the same
system or cluster. This is because both the source and target would have the same AdvFS
domain ID. The domain ID can be seen using the
advfsstat
command (Figure 8-30 on
page 448).
#cd /etc/fdmns
# ls -l DOM1
lrwxr-xr-x 1 root system 16 Sep 23 11:23 dsk58c -> /dev/disk/dsk58c
lrwxr-xr-x 1 root system 16 Sep 23 11:24 dsk59c -> /dev/disk/dsk59c
lrwxr-xr-x 1 root system 16 Sep 23 11:25 dsk60c -> /dev/disk/dsk60c
# mkdir /etc/fdmns/DOM2
# cd /etc/fdmns/DOM2
# pwd
/etc/fdmns/DOM2
# ln -s /dev/disk/dsk62c
# ln -s /dev/disk/dsk63c
# ln -s /dev/disk/dsk64c
# ls -al
drwxr-xr-x 2 root system 8192 Sep 23 14:04 .
drwxr-xr-x 13 root system 8192 Sep 23 14:04 ..
lrwxrwxrwx 1 root system 16 Sep 23 14:04 dsk62c -> /dev/disk/dsk62c
lrwxrwxrwx 1 root system 16 Sep 23 14:04 dsk63c -> /dev/disk/dsk63c
lrwxrwxrwx 1 root system 16 Sep 2314:04 dsk64c ->/dev/disk/dsk64c

448
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 8-30 Identifying AdvFS domain ID
8.7.2 Tru64 and PPRC
In this section, we discuss the steps necessary to perform PPRC remote volume replication
from a primary volume assigned to a Tru64 5.x system and how to mount the secondary on a
subsequent system. This section, like the previous one, covers both the UFS and AdvFS file
systems under Tru64.
UFS Filesystem on Tru64 and PPRC
Performing volume replication using PPRC on volumes containing UFS file systems is the
same as with FlashCopy, ensuring that you quiesce applications and unmount file systems
prior to performing split mirror operations for remote vaulting or the termination of pairs when
bringing the secondary volumes online for operation. The reason for quiescing all activity on
the primary volumes is to maintain consistency, as opposed to any restrictions on the primary
volumes.
Therefore, the process for performing volume replication using PPRC with UFS would be as
follows:
1.Establish PPRC paths between the LSSs that contain primary and secondary volumes,
taking care that there can only be a maximum of eight paths between each LSS pair.
Optionally, choose to have a consistency group relationship between the LSS pairs.
2.Establish the primary and secondary copy pair relationship. Choose the method of
synchronization, whether synchronous or asynchronous.
3.Wait for the copy pairs to change to the duplex state, indicating that data on the secondary
volume is the same as on the primary.
If you instigated a PPRC-XD relationship, then it is necessary to either perform a catch-up
operation using
go-to-sync
or to halt all updates to the primary volume so that in time all
the contents of the primary volume are reflected on the secondary.
4.Now that the volumes are in synchronous state, stop the application and terminate the
pairs.
# advfsstat DOM1

Domain -1032772981.728915- Stats -

Deref 9 Refhit 9 Refhitwait 0 R_ahead 0
Unpin 0 Pinhit 0 Pinhitwait 0 Pinreads 0
Lazy 0 Log 0 Blocking 0 Clean 0
Ubchit 9 Unconsol 0 ConsolAbort 0
UnpinMeta 0 UnpinFtx 0 UnpinData 0
DerefMeta 9 DerefFtx 9 DerefData 0

Disk Reads Writes Rglobs AveRglob Wglobs AveWglob
---- ----- ------ ------ -------- ------ --------
1 0 0 0 0 0 0
Note: Do not try to mount a FlashCopy target volume containing the AdvFS file system on
the same system or one within the same cluster, because the domain ID contained on the
volume will be identical to that on the source.

Chapter 8. Open systems specifics
449
5.If the volumes are assigned but not recognized by the system kernel, then use:
hwmgr -scan scsi
hwmgr -show scsi
6.Perform a consistency check on the secondary volume file systems:
fsck /dev/disk/dsk47c
7.Mount the file systems residing on the secondary volume:
mount /dev/disk/dsk47c /ess
AdvFS Filesystem on Tru64 and PPRC
The same rules apply with PPRC as with FlashCopy when using AdvFS file systems. It is not
possible to mount the secondary volume onto a server that is part of the same cluster as the
server on which the primary volume resides. This is due to the Domain/Fileset concepts in the
AdvFS file system, which uses a unique domain ID that resides with the volume data
structures and since the secondary is a volume level replication of the source, mounting that
onto the same server/cluster will cause conflicts.
The procedure for performing PPRC with an AdvFS file system is done as with FlashCopy:
1.Ensure that the primary and secondary volumes are identical in size.
2.Establish the paths between the primary and secondary LSSs, making sure to limit the
number of paths to eight per LSS.
3.Check and clear any persistent reservations on the secondary volumes.
4.Establish primary and secondary volume PPRC relationships in synchronous or
asynchronous (PPRC-XD) copy mode.
5.Wait for the volumes to go into duplex (synchronized) mode. If you are using PPRC-XD,
issue
go-to-sync
and wait for the secondary to catch-up. Monitor the progress of the
volumes using the Copy Services Web User Interface or the
rsQuery.sh
command.
6.Once the volumes are in sync, stop all write I/O or quiesce the application and then
terminate the copy pairs.
7.If the volumes are assigned from the ESS but not recognized by the system, issue:
hwmgr -scan scsi
hwmgr -show scsi or ./essvol
8.Manually create the AdvFS domain used for the PPRC secondary volumes under the
/etc/fdmns directory and create symbolic links to the secondary volume block special file
under /dev/disk.
9.If the secondary server is not part of the same cluster as the primary, it is possible to have
both the PPRC primary and secondary volumes mounted at the same time:
mount -t advfs DOM2#advol1 /ess/advol2
8.7.3 Managing Tru64 Persistent Reserve
In the clustered environment, Tru64 5.x hosts place a Persistent Reserve (PR) on a volume
as soon as the cluster can access it. Persistent Reserve means that no other system except
the cluster members themselves are able to access the volume.
Important: FlashCopy and PPRC will fail if the target volume contains a Persistent
Reserve.

450
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Concept of Persistent Reserve
Within the cluster environment, each cluster member creates a key entry in the Persistent
Reserve (PR) table that is held by the disk. This table can be viewed with the scu utility:
# scu -f /dev/rdisk/dskxxc show keys
One cluster member holds the reservation. Its active reservation key can also be viewed with
the scu utility:
# scu -f /dev/rdisk/dskxxc show reservations
A Tru64 cluster sets the reservation in such a way that all cluster members have read and
write access to the volume and all other hosts have read access only. All cluster members
can also modify the PR state (for example, clear the PR) using
their own
reservation key.
For example, we have a Host A that uses the key 0x10001 and Host B with the key 0x10002.
Check the active reservation using the
scu
command, as seen in Figure 8-31. Host A is the
owner of the existing reservation.
Figure 8-31 Checking for Persistent Reserve
Scenario 1
To remove the Persistent Reserve (PR) from the volume by using the host who owns it, Host
A in this example, issue the command:
# scu -f /dev/rdisk/dsk45c pres clear key 0x10001
Scenario 2
To remove the Persistent Reserve (PR) from a different host (Host B) than the one who owns
it (Host A) (for example, after the owning Host A crashed), see the command in Figure 8-32.
Figure 8-32 Trying to remove the PR from another system in the key list
The clear PR failed, because Host A used the key 0x10001; we are trying to remove the PR
from Host B, which has to use its own key. To find out the valid key of Host B, we need to
check the key table, as in Figure 8-33 on page 451. Host A uses the key 0x10001, so the key
for Host B on this volume is 0x10002.
# scu -f /dev/rdisk/dsk45c show reservations
Persistent Reservation Header:
Generation Value: 4
Additional Length: 16
Reservation Descriptors:
Reservation Key: 0x10001
Scope-Specific Address: 0
Reservation Type: 0x5 (Write Exclusive Registrants Only)
Reservation Scope: 0 (LU - full logical unit)
Extent Length: 0
# scu -f /dev/rdisk/dsk45c pres clear key 0x10001
scu: 'persistent reserve out' failed on device '2105800' at nexus [2/3/2]
SCSI Status = SCSI_STAT_RESERVATION_CONFLICT (0x18) - Target reservation conflict

Chapter 8. Open systems specifics
451
Figure 8-33 Checking the key table
If we try the same command again with the valid key, the PR will be cleared:
# scu -f /dev/rdisk/dsk45c pres clear key 0x10002
Scenario 3
In this scenario, we want to remove the PR from a different host that has no entry in the key
table. If the host has no key entry in the PR table, it cannot change the PR state of the
volume, as seen in Figure 8-34.
Figure 8-34 Trying to remove the PR from a system not in the key list
You must remove the PR using a system already listed in the key table.
# scu -f /dev/rdisk/dsk45c show keys
Persistent Reservation Header:
Generation Value: 24
Additional Length: 48
Reservation Key List:
Key Entry 0: 0x10002
Key Entry 1: 0x10002
Key Entry 2: 0x10002
Key Entry 3: 0x10002
Key Entry 4: 0x10001
Key Entry 5: 0x10001
#
# scu -f /dev/rdisk/dsk45c show reservation
scu: Unable to open device '/dev/rdisk/dsk45c', EIO (5) - I/O error
# scu -f /dev/rdisk/dsk45c show keys
scu: Unable to open device '/dev/rdisk/dsk45c', EIO (5) - I/O error

452
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments

© Copyright IBM Corp. 2001, 2003, 2004. All rights reserved.
453
Chapter 9.
Disaster Recovery
In this chapter, we discuss how to exploit ESS Copy Services for Disaster Recovery. We
analyze requirements and describe several outage scenarios.
The chapter is organized as follows:

Disaster recovery concepts

ESS Copy Services in Disaster Recovery

Types of disaster recovery procedures

Loss of Copy Services servers

Planned outages

Unplanned outages

Guidelines for Disaster Recovery

Multiple Device Manager Replication Manager

Asynchronous PPRC Utilities for Open System Environment

Enterprise Remote Copy Management Facility
This chapter helps you to apply PPRC concepts and user interfaces for disaster recovery
planning.

PPRC concepts are introduced in Chapter 4, “Peer-to-Peer Remote Copy (PPRC)” on
page 55.

User interfaces are discussed in Chapter 5, “ESS Copy Services Web User Interface prior
to LIC 2.2.0” on page 123 and Chapter 6, “ESS Copy Services Web User Interface: LIC
2.2.0 and later” on page 189

The command line interface is discussed in Chapter 7, “ESS Command Line Interface” on
page 377.

An introduction to the Multiple Device Manager Replication Manager for ESS Copy
Services is in 9.9, “Multiple Device Manager Replication Manager” on page 505.

An overview of the Asynchronous PPRC Utilities is in 9.10, “Asynchronous PPRC Utilities
for Open System Environment” on page 507.

The ESS API is discussed in Appendix A, “ESS Application Programming Interface (API)”
on page 533.

In Chapter 10, “IBM Systems Group Service Offerings” on page 509, we present
integrated solutions for the Windows platform.
9

454
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
9.1 Disaster recovery concepts
Disaster recovery planning is part of business continuity planning. For business continuity,
components like the people side of your business processes, business resumption, crisis
management, and contingency plans must be prepared too. Here we introduce only some
basic concepts. For a more comprehensive discussion, see IBM TotalStorage Solutions for
Disaster Recovery, SG24-6547.
9.1.1 What is Disaster Recovery?
How would a shutdown of your IT system affect your business? What about a site disaster?
Are your business critical processes and data protected from a site disaster? Do you put off
system maintenance and upgrades to avoid system downtime? Consider Figure 9-1.
Figure 9-1 Components of Disaster Recovery
In today's highly competitive e-business world, outages can have a devastating impact on a
business — they can even mean its demise. IBM suggests that Disaster Recovery is much
more than just mirroring the disk data; rather, as Figure 9-1 illustrates, Disaster Recovery is a
total business continuance solution comprising five major IT components:

Servers

Storage

Software and automation

Networking

Services for integration
A disaster recovery implementation that only covers the storage component will leave the
organization open to significant additional costs and time requirements if the other
components are not covered.

Operations Staff
Operations Staff
Applications Staff
Network Staff
Management
Control
Physical
Facilities
Telecom Network
Data
Operating
System
Applications
xSeries
pSeries
iSeries
Solaris
HP-UX
WinNT/2000
zSeries
zSeries
xSeries
pSeries
iSeries
Solaris
HP-UX
WinNT/2000
A comprehensive approach with the five IT
component areas results in a solution:
1. Servers
2. Storage
3. Software and Automation
4. Networking (includes Physical Infrastructure)
5. Skills and Services
The Solution: True Nature Of Disaster Recovery
Provide all
five
to assure:
"On Time, On Budget, On
Demand"

Chapter 9. Disaster Recovery
455
9.1.2 Business objectives of Disaster Recovery
As shown in Figure 9-2, to design a cost-effective solution, we start determining the following
objectives by application or business line:

Recovery Time Objective (RTO): What is the business cost-justified elapsed time to
recovery?

Recovery Point Objective (RPO): When the Recovery Time Objective is met, what
amount of data is needed to be recreated?

Network Recovery Objective (NRO): How long does it take to switch the entire network
over to the backup data center?
Figure 9-2 Business objectives of Disaster Recovery
These first two objectives (RTO and RPO) can often be balanced against each other to
optimize the cost/benefit ratio. When the third objective (NRO) comes into play, networking
issues will come into consideration, for example, there is no need to purchase a 30 minute
RTO solution if the network provider requires two hours to switch the network.
Planned versus unplanned outages
Planned outages are just as effective at removing service from the end users as unplanned
outages, and they are much more frequent. Yet, typically, disaster recovery solution cost
justification is attempted based on the unplanned outage cost alone. So a realistic return of
investment analysis should always consider the effects of planned as well as unplanned
outages.
9.1.3 Tiers of Disaster Recovery
Finally, it is not only one RTO/RPO set that you are looking for. When looking for a solution,
you will always want to know how much faster or slower the solution will be if you invest a little
Recovery Time Objective (RTO)
How long can you afford to be without your systems?
Recovery Point Objective (RPO)
When it is recovered, how much data can you afford to
recreate or lose?
Network Recovery Objective (NRO)
How long will it take to switch over the network?
Determine telecom line costs at various bandwidths.
What is your cost/recovery time curve?
If I spend a little more,how much faster is Disaster Recovery?
If I spend a little less,how much slower is Disaster Recovery?
Determining the cost vs. RTO recovery curve is the key to
Determining the cost vs. RTO recovery curve is the key to
selectin
g
the
p
ro
p
er solution
(
s
)
.
selecting the proper solution(s).
Backup
Application Failure
Application
Processing
RPO

456
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
more or less. This is the primary business issue, which leads us to the consideration of the
tiers shown in Figure 9-3.
Figure 9-3 Tiers of Disaster Recovery
The chart in Figure 9-3 is a standard Disaster Recovery industry tier chart, showing that for
varying types of RTO/RPO combinations, there are a variety of technologies.
In 1992, the SHARE user group in the United States, in combination with IBM, defined a set
of disaster recovery tier levels. This was done to address the need to properly describe and
quantify various different methodologies for successful mission-critical computer systems
disaster recovery implementations. Accordingly, within the IT Business Continuance industry,
the tier concept continues to be used, and it is very useful for describing today's disaster
recovery capabilities. They need only to be updated for today's specific disaster recovery
technologies and associated RTO/RPO.
The Seven Tiers of Disaster Recovery solutions offer a a simple methodology of how to
define your current service level, the current risk, and the target service level and target
environment.
Tier 0: No off-site data
Businesses with a Tier 0 disaster recovery solution have no Disaster Recovery Plan.

There is no saved information, no documentation, no backup hardware, and no
contingency plan.

Typical recovery time: The length of recovery time in this instance is unpredictable. In fact,
it may not be possible to recover at all.
Tier 1: Data backup with no Hot Site
Businesses that use Tier 1 disaster recovery solutions back up their data at an off-site facility.
Depending on how often backups are made, they are prepared to accept several days to
Recovery Time
Tiers based on SHARE definitions
Tiers of Disaster Recovery
15 Min.1-4 Hr..4 -8 Hr..8-12 Hr..12-16 Hr..24 Hr..Days
Tier 4 - Batch/Online database shadowing & journaling,
Point in Time disk copy (FlashCopy), TSM-DRM
Tier 3 - Electronic Vaulting, TSM**,
Tape
Tier 2 - PTAM, Hot Site,TSM**
Value
*PTAM = Pickup Truck Access Method with Tape
**TSM = Tivoli Storage Manager
*** = Geographically Dispersed Parallel Sysplex
Tier 7 - Highly automated, business wide, integrated solution (Example:
GDPS/PPRC/VTS P2P, AIX HACMP/PPRC , OS/400 HABP....
Tier 6 - Storage mirroring (example: XRC,
PPRC, VTS Peer to Peer)
Tier 5 - Software two site, two phase commit (transaction
integrity)
Applications with
Low tolerance to
outage
Applications
Somewhat Tolerant
to outage
Applications very
tolerant to outage
Tier 1 - PTAM*
Zero or near zero data
Zero or near zero data
recreation
recreation
minutes to hours
minutes to hours
data recreation
data recreation
up to 24 hours
up to 24 hours
data recreation
data recreation
24-48 hours
24-48 hours
data recreation
data recreation
Best D/R practice is blend tiers of solutions in order to maximize
application coverage at lowest possible cost . One size, one technology, or
one methodology doesn't fit all applications.

Chapter 9. Disaster Recovery
457
weeks of data loss, but their backups are secure offsite. However, this Tier lacks the systems
on which to restore data.
Tier 2: Data Backup with a Hot Site
Businesses using Tier 2 disaster recovery solutions make regular backups on tape. This is
combined with an off-site facility and infrastructure (known as a hot site) in which to restore
systems from those tapes in the event of a disaster. This Tier of solution will still result in the
need to recreate several hours to days worth of data, but it is less unpredictable in recovery
time.
Tier 3: Electronic vaulting
Tier 3 solutions utilize components of Tier 2. Additionally, some mission critical data is
electronically vaulted. This electronically vaulted data is typically more current than that which
is shipped via PTAM. As a result, there is less data recreation or loss after a disaster occurs.
Tier 4: Point-in-time copies
Tier 4 solutions are used by businesses who require both greater data currency and faster
recovery than users of lower Tiers. Rather than relying largely on shipping tape, as is
common on the lower Tiers, Tier 4 solutions begin to incorporate more disk based solutions.
Several hours of data loss is still possible, but it is easier to make such point-in-time copies
with greater frequency than data can be replicated through tape based solutions.
Tier 5: Transaction integrity
Tier 5 solutions are used by businesses with a requirement for consistency of data between
production and recovery data centers. There is little to no data loss in such solutions;
however, the presence of this functionality is entirely dependent on the application in use.
Tier 6: Zero or little data loss
Tier 6 disaster recovery solutions maintain the highest levels of data currency. They are used
by businesses with little or no tolerance for data loss and who need to restore data to
applications rapidly. These solutions have no dependence on the applications to provide data
consistency.
Tier 7: Highly automated, business integrated solution
Tier 7 solutions include all the major components being used for a Tier 6 solution with the
additional integration of automation. This allows a Tier 7 solution to ensure consistency of
data above that which is granted by Tier 6 solutions. Additionally, recovery of the applications
is automated, allowing for restoration of systems and applications much faster and more
reliably than would be possible through manual disaster recovery procedures.
Therefore, the ESS brings tremendous value to your disaster recovery solution. The right
balance needs to be found, depending on your business requirements. Figure 9-4 on
page 458 summarizes the positioning of IBM TotalStorage disk subsystems, but please
remember that the storage subsystem can guarantee only the automatic transfer of data. For
recovery, the application processes must be transferred too. If the recovery time in your case
must not exceed an hour, the integration of storage and server cluster technologies is
necessary.

458
IBM TotalStorage Enterprise Storage Server Implementing ESS Copy Services in Open Environments
Figure 9-4 Disaster recovery portfolio
When planning resources for your PPRC environmen