《《系统测试》PPT课件.ppt》由会员分享，可在线阅读，更多相关《《系统测试》PPT课件.ppt（74页珍藏版）》请在三一办公上搜索。
1、第14章 系统测试,Outline,系统测试的概要介绍线索需求规格说明的基本概念寻找线索线索测试的结构策略线索测试的功能策略系统测试指导方针,系统测试的概要介绍,系统测试是测试的三级层次中最接近日常测试实践的。它是根据预期来评估产品而不是根据规格说明或标准测试软件系统是否符合所有需求，包括功能性需求（主要依据是需求规格说明书）和非功能性需求（如可用性测试）一般由独立测试人员（测试小组）执行，通常采用黑盒测试方式,系统测试之功能性测试,目标检查应用程序是否达到了它预期的要求判断依据行为/功能说明书测试用例一系列的原子系统功能（Atomic System Function,ASF）,系统测试之压力
2、测试,push it to its limit+beyond,Application(System),:,Users,Volume,Resources:phy.+logical,response,系统测试之性能测试用户的角度观察delay,throughput系统的支撑强度memory,、CPU等系统测试之可用性测试系统操作过程中人的要素GUI,messages,reports,线索（thread）,线索的多种看法一般使用的场景系统级测试用例激励/响应对由系统级输入序列产生的行为端口输入和输出事件的交替序列系统状态机描述中的转换序列根据系统级线索看系统测试,对象消息和方法执行的交替序列机器指令
3、序列源指令序列MM-路径序列原子系统功能序列,线索的层次,线索有不同的层次单元级通常被理解为源指令执行时间路径，或DD-路径集成级为MM-路径，即模块执行和消息交替序列系统级是原子系统功能序列由于ASFs在输入和输出时有端口事件，因此ASFs包含端口输入和输出事件的交替序列线索提供三层测试的统一视图单元测试进行单个函数测试单元之间集成测试检查交互系统测试检查原子系统功能间的交互,线索定义,（原子系统功能，ASF）定义ASF是一种在系统层可以观察到的端口输入和输出事件的行动ASF开始于一个端口输入事件，遍历一个或多个MM-路径的一部分，以一个端口事件结束举例在SATM系统中，数字输入、现金给付和
4、会话关闭都是ASF的好例子,线索定义（续）,ASF表示集成测试与系统测试间的缝隙，是集成测试的最大测试项，是系统测试的最小测试项，因此可以在两个级别上测试ASF举例-数字输入ASF系统测试过程端口输入事件是由keysensor检测到的按下物理键，并作为字符串变量发送给GetPIN。GetPIN确定是否按下数字键或取消键，并做出相应的应答。ASF最后显示屏幕2或4集成测试过程不要求系统输入键并在屏幕上显示，而是使用驱动器提供的这些功能，并通过集成测试检查数字输入ASF,线索定义（续）,ASF图（通过ASF定义的系统）是一种有向图，其中的节点表示ASF，边表示串行流源ASF/汇ASF是一种原子系统
5、功能，在系统ASF图中作为源/汇节点出现例子-SATM系统“ATM卡输入”源ASF；会话结束ASF-汇ASF,线索定义（续）,系统线索在系统的ASF图中，是一条从源ASF到汇ASF的路径线索图（给定一个通过系统线索定义的系统）是一种有向图，节点表示系统线索，边表示单个线索的顺序执行上述这组定义提供了线索更广的视图的内聚集，以非常短的线索开始（一个单元内），以系统级线索之间的交互结束,需求规格说明的基本概念,目标讨论系统测试级别上的一组基本需求规格说明的构造五种基本需求规格说明的构造组成DataActionsPortsEventsThreads每个系统都可以用上述5个基本概念表示为了说明它们是如
6、何支持测试人员的线索标识过程,以数据为中心的线索标识,在以数据描述的系统中关注的是系统所使用和创建的信息。采用变量、数据结构、字段、记录、数据存储和文件来描述数据举例：E/R模型是最高层数据描述的最常见的选择以数据为中心的观点还是许多面向对象方法的切入点数据指经过初始化、存储、更新或可能销毁的信息举例-SATM系统初始数据描述各种帐户（PAN）及其PIN，每个帐户都有一个数据结构，包含诸如帐户余额这样的信息,以数据为中心的线索标识（续）,以数据为中心的系统常常以CRUD行动开发（“Create,Retrieve,Update,Delete”）有时线索可直接通过数据模型标识。数据实体之间的关系可
7、以是1：1、1：n等，这些差别在处理数据的线索中都有应用举例-银行客户可拥有多个帐户，每个帐户需要唯一的PIN。如果多人可以访问同一个帐户，则需要具有相同的PAN的ATM卡,以数据为中心的线索标识（续）,还可能有只读取但从不写入的初始数据（如PAN和“预期PIN”对偶）这种只读数据必须是系统初始化过程的一部分如果不是，那么必须有创建这种数据的线索因此，只读数据是一种源ASF指示器,以行动为中心的线索标识,以行动为中心建模是需求规格说明的一种常见形式行动有输入和输出，这些输入和输出可以是数据，也可以是端口事件行动同义词：转换、数据转换、控制转换、处理、活动、任务、方法和服务行动还可分解为底层活动
8、（如数据流图）行动的输入/输出视图是功能性测试的基础，行动的分解（以及最终实现）则是结构性测试的基础,以端口为中心的线索标识,每个系统都有端口（和端口设备）这些端口设备是系统级输入和输出的源和目的地区分端口和端口设备间的微小差别技术上，端口是I/O设备接入系统的点如串行端口、并行端口、网络端口、电话端口如果没有实际端口设备，系统测试可以通过“将端口边界向内移动”到端口事件的逻辑实例上实现用术语“端口”代替端口设备如SATM系统中端口：数字、取消键、功能键、显示屏幕、存款和取款通道、ATM卡和收据槽考虑端口有助于测试人员定义功能性测试和系统测试所需的输入空间,以事件为中心的线索标识,有数据和行动
9、方面的一些特征事件是发生在端口设备上的系统级输入（或输出）事件也可以是行动的输入和输出事件可以是离散（如SATM键盘输入），也可以是连续（如温度、高度或压力）离散事件必须有一定的持续时间，这在实时系统至关重要可把输入事件看作是破坏性读出数据，输出事件看作是破坏性写入数据,以事件为中心的线索标识（续）,事件是物理事件和这些事件的内部逻辑表示的转换点端口输入事件是物理到逻辑的转换端口输出事件是逻辑到物理的转换举例-SATM系统当显示屏幕5时，按B1键的端口输出事件表示“余额”，当显示屏幕6时，表示“检查”这叫做“与语境有关的端口事件”,线索,线索是5种基本构造中最不经常使用的因为要测试线索，因此测
10、试人员通常不能在数据、事件和行动之间的交互中找出线索通常在控制模型中容易找出线索发现线索系统的有限状态机模型是发现线索的一个良好的起点，因为这些路径很容易转换为线索,基本概念的E/R模型,input,output,采用基本概念建模,三种基本形式的需求规格说明模型,采用基本概念建模（续）,结构模型用于开发，表示功能分解、数据分解和组件之间的接口语境模型常常是结构模型的开始点，强调系统设备，也比较强调行动，非常间接地关注线索行为模型（控制模型）将5种基本构造中的四种集成到一起。选择适当的控制模型是需求规格说明的基础太弱的模型不能表示重要的系统行为，太强的模型一般会淹没没有意思的行为一般地，决策表只
11、对计算系统是个好选择，有限状态机对于菜单驱动的系统良好，petri网是针对并发系统的模型,寻找线索,Test threads,The macro level states are:Card entryPIN entryTransaction request(and processing)Session managementNote that this stated order is the same as the testing order since these stages are in prerequisite order.(We cant enter a PIN until succe
12、ssfully card entry),Preconditions,We need some precondition data:,Contd,SATM is initially displaying screen 1.The total cash available to the withdrawal dispenser is$500We will express threads in tables in which pairs of rows correspond to port inputs and expected port outputs.Lets start with 3 basi
13、cs threads,one for each transaction type.,Balance inquiry,Deposit,Withdrawal,Invalid card,This is the shortest thread,it consists of an invalid card,which is immediately rejected.,Variation of PIN entry,Following the macro state along thread 1,we next perform variations on PIN entry.We get 4 threads
14、,which yield edge coverage in the PIN entry finite state machine,PIN entry,PIN entry(contd),Transaction request,We have variations regarding the three types of transactions,the two types of accounts,and the amount requested.Now we focus on the amount driven-threads.,Wrong amount,Rejection the attemp
15、t to withdraw an amount not in$10 increments.,Over withdrawal,Attempt to withdraw more than the account balance.,Dispenser limit,Attempt to withdraw more cash than the dispenser contains,Session management,Deposit failure,Notice that we didnt use screen 12,which informs the user that deposits cant b
16、e processed.This is an example of a thread selected by a precondition that is hardware failure.Lets just give it a name:thread 13.,Threads for context sensitive input events,Objective:exercise context sensitive input events.Note:Some of the 13 threads exercise context sensitivity.,Context sensitive
17、threads,Contd,These 22 threads comprise a reasonable test of the SATM system.Of course,certain aspects are untested.Example:Amount entry process,the possibility of a Cancel keystroke at any point during amount entry produces a multiplicity greater than that of PIN entry,线索测试的结构策略,生成线索测试用例很容易，但确定实际使用
18、哪个测试用例则更复杂在系统级同样存在单元级的路径爆炸问题,线索测试的结构策略-自底向上组织线索,当在层次结构中组织状态机时，可以自底向上进行举例在“PIN输入尝试”状态机中，共有6条路径。遍历这6条路径要测试三件事：正确识别并回显所输入的数字、响应取消键入和匹配预期PIN和所输入的PAN。一旦测试了这个部分后，可以上升到“PIN输入”状态机，共有4条路径,线索测试的结构策略-自底向上组织线索（续）,正如在单元测试中所见，结构性测试可能会产生误导假设是路径遍历会发现缺陷，遍历各种路径会产生冗余这些线索还有一个更严重的问题：由于是分层状态机，因此实际上这些线索不能单独执行,节点与边覆盖指标,由于有
19、限状态机是有向图，因此可以使用与单元级相同的测试覆盖指标层次结构关系指出上层状态机必须把下层状态机作为输入和返回过程处理两种基本的选择是节点覆盖和边覆盖节点覆盖类似于单元级的语句覆盖边（状态转移）覆盖是更可接受的标准。如果状态机“构建良好”（通过端口事件转换），则覆盖也可保证端口事件覆盖,线索测试的功能策略,如果一个系统没有行为模型（即FSMs）有两种选择开发行为模型进行功能性测试在系统级上的类比在标识功能性测试用例时，我们使用了输入和输出空间信息，以及功能本身采用通过三个基本概念导出的覆盖指标描述功能线索事件、端口和数据,线索测试的功能策略-基于事件的线索测试,考虑端口输入事件的空间。五个端
20、口输入事件覆盖指标：PI1:each port input event occurs PI2:common sequences of port input events occur PI3:each port input event occurs in every“relevant”data context PI4:for a given context,all“inappropriate”input events occur PI5:for a given context,all possible input events occur,处理与语境有关的端口输入事件，是物理输入事件,端口输出事
21、件定义的覆盖指标,PO1:each port output event occurs.PO1 is acceptable minimum especially when a system has a rich variety of output messages for error conditions.PO2:each port output event occurs for each cause.PO2 is good but hard to quantify Basically it refers to threads that interact with respect to a po
22、rt output event.Usually a given port output event has a small number of causes.,基于端口的线索测试,是基于事件测试的有用补充通过基于端口的测试，对于每个端口都要询问端口上会出现什么事件根据每个端口的事件列表寻找使用输入端口和输出端口的线索对于端口设备来自外部提供商的系统特别有用从E/R模型可看出，设备和事件之间的多对多测试应该在两个方向上进行基于事件的测试覆盖从事件到端口的一对多关系，反之，基于端口的测试覆盖从端口到事件的一对多关系,基于数据的线索测试,基于端口和基于事件的测试适合主要以事件驱动的“反应式”系统因为
23、这些系统要对激励（端口输入事件）做出响应，并且反应常常以端口输出事件的形式做出反应式系统的两个特征：长时间运行和维持与环境的关系一般地，事件驱动的反应式系统没有有意思的数据模型，因此基于数据模型的线索不是特别有用,基于数据的线索测试（续）,非反应式系统是典型的“静态”，因而是转换式系统支持以数据库为基础的事务处理，一般采用E/R 模型描述定义基于数据的覆盖指标 描述线索集合关系中的信息一般包含在系统级线索中，而实体中的线索一般局限于单元级,基于数据的线索测试（续）,定义下列覆盖：DM1:exercise the cardinality of every relationship.DM2:exe
24、rcise the participation of every relationship.DM3:exercise the functional dependencies among relationships,DM2:ensure where or not every instance of an entity participates in a relationship.Some modeling techniques express participation in numerical limits(i.e.OMT,“at least one and at most 12”).When
25、 available,this information leads to boundary value system threads.DM3:it may be possible to determine explicit logical connections among relationships=functional dependencies as in relational databases.These are reduced when the database is normalized,but they still exist and lead to interesting te
26、st threads.,系统测试指导方针,伪结构系统测试运行剖面累进测试和回归测试,伪结构系统测试,At the system level we can use graph based metrics as a cross check on the functional threads.The claim is for pseudo-structural testing since the node and edge coverage metrics are defined in terms of a control model of a system and not directly der
27、ived from the system implementation.Behavioral models are only an approximation to the systems reality:why it is possible to decompose the models to several levels of detail.A true structural models sheer size and complexity would make it too cumbersome to use.,Decision tables and FSMs are good choi
28、ces for ASF testing.For example:If an ASF is described using a decision table,conditions usually include port input events,and actions are port output events It is possible to devise test cases that cover every condition,every action,or most completely,every rule.As in FSMs,test cases can cover ever
29、y state,transition or every path.,运行剖面,Many aspects of testing can be related to the old 80/20 rule:for a system with many threads,80%of the execution traverses only 20%of the threads.The basic concept in testing:execute test cases such that when a failure occurs,the presence of a fault is revealed.
30、The distribution of faults is only indirectly related to the reliability of the system.,Simple view of reliability:The probability that no failures occur during a specific time interval.If faults are in less traveled threads of system,then reliability will appear higher than if the same number of fa
31、ults were in the high traffic areas.Operational profiles:Determine the execution frequencies of various threads,then select threads accordingly.Operational profiles maximize the probability of finding faults by inducing failures in the most frequently traversed threads.,System Testing Categories,Obj
32、ective of system testing:To verify whether the implementation(or system)conforms to the requirements as specified by the customer(s).To verify whether the system meets a wide range of unspecified expectationsSystem testing is performedAfter constructing a reasonably stable system in an emulated envi
33、ronment.In the real environment(if the real environment is not accessible,system testing is done using models in an emulated environment)It is important to categorize the kinds of system tests for a number of reasons:Systematically focus on different aspects of a system while evaluating its quality.
34、Test engineers can prioritize their activities.Planning based on test categorization has the advantage of obtaining a balanced view of testing.Testing categories(11 categories),Basic System Tests,These provide evidence that the system can be installed,configured and brought to an operational state.B
35、asic tests are performed to ensure that commonly used functions,not all of which may directly relate to user-level functions,work to our satisfaction.The following are the major categories of subsystems whose adequate testing is called basic test.a.Boot tests:verify that the system can boot up its s
36、oftware image from the supported options(ROM,PCMCIA,.)b.Upgrade/downgrade tests:verify that the software image can be upgraded or downgraded(rolled back)in a graceful manner.c.Light emitting diode(LED)tests.These are designed to ensure that that visual operational status of the system is correct.d D
37、iagnostic tests.These are designed to ensure that the hardware components of the systems are functioning as desired.(Power-on self test(POST),Memory,Address and data buses,Peripheral devices)e)CLI(command line interface)tests.Ensure that the system can be configured.Ensure that uses commands are pro
38、perly interpreted.Verify the error message,Functionality Tests,Verify the system as thoroughly as possible over the full range of requirementsLogging and tracing tests:(implicit functionality)GUI tests:(Icon,Menu bar,Dialog box,Scroll bar)Security tests.Verify that the system meets the requirements
39、for detecting security breaches and protecting from such breaches(Unauthorized access,Illegal file access,Virus),Robustness Tests,To verify how gracefully the system behaves in error situations,or how it handles a change in its operational environment.Different kinds of tests are:Boundary value test
40、s(valid and invalid inputs)Recover from power failureOn-line insertion and removal(OIR)System recovers after an OIR eventAvailability of redundant modulesDegraded node test(a portion of the system fails),Interoperability and Performance Tests,Interoperability tests(3rd party products):Verify that th
41、e system can inter operate with 3rd party products.Performance tests:Determine how actual system performance compares to predicted performanceResponse timeExecution timeThroughputResource utilizationTraffic volume,Scalability Tests,Verify that the system can scale up to its engineering limitsData st
42、orage limitations(counters and buffers)Speed limitations(CPU)Communication limitationsResource intensive,Stress Tests,Stress tests(push it over the edge to break it).Evaluate the behavior of a software component when offered load is in excess of its designed capacityPush the system“over the edge”and
43、 observe that the recovery mechanism worksBring out the following kinds of problems:Memory leaksBuffer allocation problem,Load and Regression Tests,Load and stability tests:Verify that the system can operate in a large scale for a long time(months)Regression tests:Ensure that nothing had has happene
44、d after a fix.Five possibilities can happen after an attempt to fix.fix the bug reportedfail to fix the bugfix the bug,but break something elsefail to fix the bug,but break something elsefix this bug and fix some unknown bugs,Documentation Tests,Documentation tests:This is a review of technical accu
45、racy and readability of user manuals includingtutorialson-line helpThere are three kinds of documentation testsRead test(clarity,organization,flow and accuracy)Hands-on test(evaluate usefulness)Functional test(verify the document),Conformance to Regulatory Bodies,Conformance to regulatory bodies:Identify unsafe consequencesFederal Communication Commission(FCC)and Canadian StandardsAssociation(CSA)certify a products safety.,
