《大规模并行处理机系统.ppt》由会员分享，可在线阅读，更多相关《大规模并行处理机系统.ppt（41页珍藏版）》请在三一办公上搜索。
1、第7讲 1 大规模并行处理机系统 MPP,古志民,千万亿次超级计算机-天河一号Tianhe-1 2009,天河一号特点,我国首台千万亿次超级计算机系统“天河一号”由国防科学技术大学研制成功。在今天中国高性能计算机TOP100组织公布的2009年度前100强排名中，天河一号高居榜首。有关专家认为，“天河一号”的诞生，是我国战略高技术和大型基础科技装备研制领域取得的又一重大创新成果，实现了我国自主研制超级计算机能力从百万亿次到千万亿次的跨越，使我国成为继美国之后世界上第二个能够研制千万亿次超级计算机系统的国家。系统峰值性能达每秒1206万亿次双精度浮点运算，内存总容量98TB，点点通信带宽每秒40
2、Gb，共享磁盘容量为1PB，具有高性能、高能效、高安全和易使用等显著特点，综合技术水平进入世界前列.,IBM千万亿次超级计算机,IBM千万亿次计算的超级计算机,IBM为美国洛斯阿拉莫斯国家实验室建造的计算机系统成为世界上首个突破每秒钟一千万亿次计算的超级计算机。排名前10名中有5个系统出自IBM；前50名中有17个系统出自IBM；前100名中有35个系统出自IBM，此外，上海超级计算中心的“曙光”5000A排名第15位。在500强榜单中，有188台超级计算机来自于IBM，却有212台超级计算机来自惠普。IBM for los national laboratory building compu
3、ter system become the worlds first breakthrough one quadrillion times per second calculation of the super computer.The top 10 has five system from IBM;Top 50 has 17 system from IBM;In the first 100 has 35 system from IBM,in addition,Shanghai supercomputing center dawn 5000 a ranking 15th.In the 500
4、list,there are 188 sets of super computer from IBM,are 212 supercomputer from HP.,1MPP(massively parallel processing),MPP(massively parallel processing)is the coordinated processing of a program by multiple processor s that work on different parts of the program,with each processor using its own ope
5、rating system and memory.Typically,MPP processors communicate using some messaging interface.In some implementations,up to 200 or more processors can work on the same application.An interconnect arrangement of data paths allows messages to be sent between processors.Typically,the setup for MPP is mo
6、re complicated,requiring thought about how to partition a common database among processors and how to assign work among the processors.An MPP system is also known as a loosely coupled or shared nothing system.An MPP system is considered better than a symmetrically Multi-processing system(SMP)for app
7、lications that allow a number of databases to be searched in parallel.These include decision support system and data warehouse applications.,2 MPP Architecture,高速网络（HSN）,本地互连网络,NIC,P/C,.,P/C,M,磁盘和其他I/O,SMP/SINGLE PROCESSOR,MPP with/without SMP,SMP2-64 processors todayShared-everything architectureAl
8、l processors share all the global resources availableSingle copy of the OS runs on these systemsMPP A large parallel processing system with a shared-nothing architectureConsist of several hundred nodes with a high-speed interconnection network/switchEach node consists of a main memory&one or more pr
9、ocessorsRuns a separate copy of the OS,3 可扩放性scalability-If an application needs more MIPS or megabytes,additional processors can be added help solve the problem,采用物理分布式主存结构distributed memory system；平衡的处理能力和主存与I/O能力，保证数据快速送入处理器；平衡的计算能力和并行性以及交互能力，保证进程/线程管理及通信与同步极小的开销；以上述条件为基础实施可扩放性。In a massively par
10、allel processing system,current levels of technology allow for Thousands of processors per system Tens/Hundreds of Megabytes of RAM per processor Gigabytes of disk storage per processor Tens of Megabytes/sec global communication bandwidth per processor Hundreds of MIPS/MFLOPS per processor,4 系统成本Sys
11、tem Cost,需要控制MPP系统中每一部件成本，采取的措施：利用Moor定律（每1824个月性能就翻一番）选用商用微处理器（为PC或小型系统或工作站设计）；采用壳体系结构（用shell方法，系统其他部分无须改变），支持（微处理器）部件换代的可扩放性；然而也产生了问题：物理地址空间太小；TLB（Translation Look-aside Buffer）太小；单字（Single-Word Stride）存取效率很低等。The need to control MPP system in each unit cost,take measures:1 Use Moor law(every 18 2
12、4 months performance is doubling)choose commercial microprocessor(for PC or a small system or workstation design);2 The shell system structure(with shell method,system other part does not need to change)support(microprocessor)unit scalability;However also produced a problem:physical address space is
13、 too small;TLB(Translation Look-aside Buffer)is too small;Words(Single-Word Stride)access efficiency is very low.,5 通用性和可用性,支持MIMD；支持PVM、MPI、HPF；支持节点分区；高可用性；其他：支持通信需求；支持可扩放I/O性能；,面临的问题(some difficulties),实际性能差:Rmax Rpeak；并行程序可编程性困难,need new programming tools；If the system is designed intelligently,t
14、he overall performance of the system(global communication bandwidth,MIPS,MFLOPS,etc.)will scale up linearly with the system size.It should be noted,though,that the degree to which performance can be extracted from a MPP system is very algorithm dependent.Undoubtedly the level of computing power avai
15、lable in a large MPP system will increase dramatically over time.Processor speeds and memory sizes are doubling approximately every eighteen months and this increase will be quickly adopted by MPP manufacturers.This means that the age of a Teraflop/Terabyte computer is not far off.Extremely large am
16、ounts of data will be able to be analyzed using this amount of processing power.,7 实例1：Cray T3E体系结构NCC-NUMA+DSM,三维双向环网链接,I/O设备,千兆环通道,Alpha21164,主存,控制和寄存器,路由器,shell,8 Cray T3E性能,300 MHZ ProcessorEach processor Rpeak=600Mflops62048 processorsSystem Rpeak=3.61228GflopsMemory size=14096GBMemory Rpeak=7.
17、22450Gb/sNetwork Rpeak=600MB/s,9 T3E系统软件与价格,UNICOS/mk(64 bit UNIX)PVMMPIHPFC/C+Totalview并行程序调试器MPP Apprentice并行性能分析工具100万美元，1995年交付使用。,10 实例2 Intel/Sandia ASIC Option Red（1997年交付使用，NORMA结构）,4608节点，其中COMPUTE NODES 4536，SERVICE NODES 32，I/O NODES 24，SYSTEM NODES 2，BACKUP NODES 14；1540 POWER；616 MAINBO
18、ARD；640 DISKS；2 个200MHz Pentium Pro处理器/每个NODE；594 GB Memory,11 Intel/Sandia ASIC Option Red（Architecture of Mesh Routing Component）,MRC,MRC,MRC,MRC,MRC,MRC,MRC,MRC,MRC,MRC,MRC,MRC,NIC OF MAINBOARD,12 双节点（4CPU）主板结构,L2 CACHE P6,NIC,L2 CACHE P6,L2 CACHE P6,L2 CACHE P6,NIC,引导支持,主存控制,主存控制,SIMMS,SIMMS,I/O
19、桥,扩展连接器,I/O桥,引导支持,扩展连接器,64bit，66MHzLocal bus,ICF,PCI bus,13 单节点（2 CPU）主板结构,PCI,L2 CACHE P6,L2 CACHE P6,NIC,引导支持,主存控制,主存控制,SIMMS,SIMMS,I/O桥,扩展连接器,I/O桥,引导支持,扩展连接器,64bit，66MHzLocal bus,ICF,PCI bus,14 ASCI Option Red系统图,DISK,DISK,DISK,PCI NODE,COM.NODE,PCI NODE,PCI NODE,COM.NODE,COM.NODE,COM.NODE,COM.NO
20、DE,COM.NODE,SER.NODE,SER.NODE,SER.NODE,PCINODE,ETHE.NODE,NODE站（SSI）,引导NODE,I/O,COMPUTING.NODE,I/O,服务,SYS.NODE,What is Single System Image(SSI)?,A single system image is the illusion(幻觉),created by software or hardware,that presents a collection of resources as one,more powerful resource.SSI makes th
21、e MPP/cluster appear like a single machine to the user,to applications,and to the network.,15 系统软件,Paragon(Based OSF UNIX)For Compute Node run Cougar(Light Weight Kernel)MPINX Message LibC/C+,MPP Network Review,MPP Network Review,Multithreading,without multithreading support,with multithreading supp
22、ort,A related model to SIMD is vector processing-GOODYEAR MPP,1983,MIMD-IBM RS/6000 SP2 with 256 processors.This distributed-memory machine is built using boards from desktop computers largely unchanged plus a custom switch as the interconnect.Photo courtesy of the Lawrence Livermore National Labora
23、tory.,Scalability Vs.Single System Image,UP,16 机群（Cluster）系统引子,计算机机群CLUSTER OF COMPUTER,A cluster is a type of parallel or distributed processing system,which consists of a collection of interconnected stand-alone computers working together as a single,integrated computing resource.a collection of w
24、orkstations of PCs that are interconnected by a high-speed networkwork as an integrated collection of resources have a single system image spanning all its nodes,计算机机群系统结构Architecture of Cluster,高速互连网络HSN,机群中间层 SSI、可用性底层,OSNODE,OSNODE,OSNODE,OSNODE,OSNODE,串行应用,并行应用,并行编程环境PVM、MPI、Java,Computer Cluste
25、r by Using Network,计算机机群连接方式1（无共享）,D,P/C,M,MIO,NIC,D,P/C,M,MIO,NIC,LAN,计算机机群连接方式2（共享磁盘）,D,P/C,M,MIO,NIC,D,P/C,M,MIO,NIC,共享磁盘,19 连接方式（共享存储器）,D,P/C,M,MIO,NIC,D,P/C,M,MIO,NIC,SCI,21 设计要点,可用性：充分利用冗余资源，使系统在尽可能时间内为用户服务；单一系统映像SSI：通过组合各节点OS提供对系统资源的统一访问；Job ManagementPFS需要高效通信系统,关于可用性中的检查点问题 CHECKPOINT(a,b,c
26、),可在内核、库、应用程序三级发生；,a,b,d,c,x,y,z,P,Q,R,Process,一致性快照Checkpoint Consistency Snapshot（a-Consistency一致;b-No Consistency不一致）,如果进程之间不存在一个进程的检查点已接收了消息，而另一进程的检查点还未发送这个消息。称一致性快照。,a,b,x,y,z,P,Q,R,C?,If there is not the process which receives a message at checkpoint and another process has not sent the messag
27、e at checkpoint,then we say that the related checkpoints are the Consistency Snapshot,homework,1.What are the differences about MPP and SMP?Answer:MPP A large parallel processing system with a shared-nothing architectureConsist of several hundred nodes with a high-speed interconnection network/switc
28、hEach node consists of a main memory&one or more processorsRuns a separate copy of the OSSMP2-64 processors todayShared-everything architectureAll processors share all the global resources availableSingle copy of the OS runs on these systems2.What is SSI?,HOMEWORK,1.Which is the Consistency Snapshot for a、b、c and d？,a,b,c,d,x,y,z,P,Q,R,m,
