Re: ST evaluation on threats and security objectives

[Gary Stoneburner <garystoneburner@comcast.net>]

The CC allows ST/PP authors to (1) assume threats away by an assumption 
that the environment takes care of it or (2) list the threats in the 
environment section and have environmental objectives to address them.

It is fully the choice of the author as to how detailed the threat 
description is in a PP or ST.

There is, presently, no difference in the evaluation of a PP at EAL-1 for a 
TOE that claims to do almost nothing and a PP at EAL-7 for a TOE essential 
to national security.  My personal opinion is that this is a serious 
error.  Yet this is the current situation.

Cheers,
Gary

At 06:24 2/24/03, YOKOTA HIROFUMI wrote:
>Thanks Gary,
>
>I think, your mentioned fundamental issue is true.
>Obvious missing-threats should be captured by ST evaluator.
>
>However, the issue is not only related to the capabilities of evaluator,
>  but also related to the level of descriptions of threats and assumptions
>  in the security environment that ST authers provide.
>
>I mean:
>if the threats and assumptions are not described in detail and specifically,
>  it would be hard for the ST evaluator to find obvious missing threats.
>
>This is well mentioned in the RI#187 (Threats and Attacks) as follows.
>----------------------------------------------------
>It is acknowledged that too much detail is inappropriate because
>  it may leave out valid attack and existing vulnerabilities and provide
>  unintended information to a potential threat agent. However, reducing
>  the threat set to the minimum is also not satisfactory since it provides
>  essentially no information to the potential user of the PP (or ST)
>  regarding the intent and capabilities of the TOE.
>----------------------------------------------------
>Although, the above statements are mentiond for threats,
>  I think, the same thing is true for assumptions.
>
>I observe the following generic assemptions in several evaluated STs.
>
>- The TOE is physically secure.
>- The authorized user is non-hostile and follow all usage guidance.
>- The system is adequately physically protected against loss of communications
>    i.e., availability of communications.
>- The abstruct machine of the TOE operates in a correct and expected manner.
>- The TOE and the TOE environment are competently installed and administered
>    to allow for correct operation of the TOE.
>- The computer system associated devices and equipment function correctly.
>
>It appears that those assumptions are simply declaring that TOE environment
>  is secured and there is no threat there.
>It would be hard to find missing threats with those assumptions, since 
>threats
>  are simply excluded.
>
>On the other hand, there is another level of description like the following.
>
>- The TOE does not directly connect to a public network. The TOE is 
>located on a
>    network that is properly protected by a firewall, or on a network that 
> has no
>    connections to a public network. Therefore, it is not assumed that unknown
>    malicious users attack directly to the TOE in a sophisticated manner 
> in the
>    firewall-protected network, e.g. attacks utilizing enormous number of 
> machines.
>
>I think, this level of detail meets CEM ASE_ENV.1-1[318] guidance, which 
>requests
>  the environment of use of the TOE is explained in sufficient detail.
>
>However, I do not expect ST authors of lower assurance level of EAL1-EAL2
>  are verbose like this.
>
>I agree that finding obvious missing threats at the ST evaluation is important
>  and should not be underestimated.
>
>However, I think, it would be only practical with the proper level of 
>statement
>  of threats and assumptions in STs.
>It would be hard for STs of lower assurance level, since ST authors are 
>likely to
>  exclude threats, by simply declaring that TOE environment is 
> secured..    .
>
>Regards,
>    Yokota
>
>---- Original Message -----
>From: <mailto:gary.stoneburner@nist.gov>Gary Stoneburner
>To: <mailto:cc-cmt@nist.gov>Multiple recipients of list
>Sent: Friday, February 21, 2003 11:52 PM
>Subject: Re: ST evaluation on threats and security objectives
>
>You ask "So, examining the lacking of threats and insufficiencies of the 
>security objectives
>would mostly be the work of the TOE evaluation.  Wouldn't they?"
>
>The CC uses the phrase "TOE evaluation" to mean the evaluation of the ST 
>and the evaluation of the TOE.  So the answer to the intent of your 
>question is no.  While it is true that ST issues may be uncovered during 
>the evaluation of the TOE against the ST.  That is why the evaluation of 
>the ST is not considered complete until after the TOE has been evaluated 
>against it.  Yet a missing threat or insufficient security objective is 
>not the type of thing the I would expect the evaluation of TOE to ST would 
>uncover - other than just because of additional looking at the ST.
>
>I think that the fundamental issue here is the need to differentiate 
>between what is required to be successfully evaluated, considering 
>practical realities not hoped for evaluator capabilities, and what 
>additional things would be of value to users of the PP or ST.
>
>    For the former, we just cannot expect the evaluation process to 
> uncover all missing threats.  The evaluator may see some of the more 
> obvious ones, but that is about the extent of reasonable 
> expectations.  And the CC does not even require an expression of the risk 
> acceptance that is implied in the ST.
>
>    For the latter (user need), it would be fantastic if effort was made 
> by the PP/ST author team to show a complete threat/policy list and to 
> capture all of the assumptions made in developing the PP/ST.  It would be 
> fantastic if the authors purposefully addressed risk acceptance made this 
> explicit.  Explicit risk acceptance would even be of benefit to 
> evaluators, helping them determine that the objectives are sufficient.
>
>By the way, NIST is working on a CC Users guide that will help potential 
>users of the CC see the value in CC, PP, and ST that goes beyond what is 
>required by or should be expected of an evaluation.
>
>Cheers,
>Gary
>
>At 07:27 AM 2/21/2003, YOKOTA HIROFUMI wrote:
>>Thanks Gary,
>>
>>I agree with you that inconsistency should cause fail evaluation of the ST.
>>
>>However, as you say, it is more likely that the ST/PP is not obviously
>>  inconsistent, regarding the lacking of threats and insufficiency
>>  of security objectives.
>>
>>I prefer to understand this like the following:
>>
>>The ST evaluator performs an internal consistency analysis
>>  to ensure that the ST does not include ambiguities and does
>>  not include contradictory statements.
>>
>>If inconsistencies are found, the ST evaluation should fail.
>>
>>However, the ST evaluator should not be required to examine
>>  the completeness of threats or the sufficiency of security objectives
>>in the ST,
>>
>>Because, selecting and defining what threats and what security objectives
>>  are required is the developer's design decision based on their risk 
>> analysis
>>  and valunerability analysis.
>>
>>The ST evaluators would not be prepared for the time and effort
>>  to examine the completeness and the sufficiency.
>>
>>However, instead, the TOE evaluation would do this.
>>
>>CEM(AVA_VLA.1-1) says:
>>The evaluator shall examine the developer's vulnerability analysis
>>  to determine that the search for obvious vulnerabilities has considered 
>> all
>>  relevant information.
>>
>>CEM(AVA_VLA.1-10) say:
>>The evaluator shall report in the ETR all exploitable vulnerabilities and
>>  residual vulnerabilities, detailing for each:
>>b) the implicated security function(s), objective(s) not met,
>>  organizational security polic(ies) contravened and threat(s) realised;
>>
>>*****
>>
>>So, examining the lacking of threats and insufficiencies of the security 
>>objectives
>>would mostly be the work of the TOE evaluation.
>>
>>Wouldn't they?
>>
>>----- Original Message -----
>>From: <mailto:gary.stoneburner@nist.gov>Gary Stoneburner
>>To: <mailto:cc-cmt@nist.gov>Multiple recipients of list
>>Sent: Friday, February 21, 2003 1:18 AM
>>Subject: Re: ST evaluation on threats and security objectives
>>
>>I hope the ASE supplement maintains the requirement for ST internal 
>>consistency and correctness.  If so, the treats listed must be consistent 
>>(or at least not inconsistent) with the problem description captured in 
>>the ST introduction and the TOE Description.
>>
>>For example, if the introduction and TOE description indicate that TOE 
>>connects to the Internet and is expected to protect itself, then threats 
>>related to Internet attackers must be included for the ST to be 
>>internally consistent and correct.  An ST lacking such threats in the 
>>Security Environment section should fail evaluation.
>>
>>The evaluation decision that the ST (or PP) is internally consistent is 
>>actually more a determination that the ST/PP is not obviously 
>>inconsistent.  This determination includes a subjective component and is 
>>dependent on the specific experience and capabilities of the evaluation team.
>>
>>Cheers,
>>Gary
>>
>>At 05:54 AM 2/19/2003, YOKOTA HIROFUMI wrote:
>>
>>>Are "risk analysis and vulnerability analysis" subject to the ST evaluation?
>>>
>>>I mean:
>>>Should the ST evaluator issue fail verdict on account
>>>of finding lack of threats or week point of security objectives ?
>>>
>>>Thankfully, Supplement(ASE) is clear about this.
>>>
>>>It says:
>>>- a risk analysis is not subject to the CC,
>>>- and, therefore, there is no requirement for the security problem
>>>   definition to be complete.
>>>- the ST evaluator does need to evaluate the security objectives
>>>   not for the suitability to counter the threats, but for covering all
>>>   identified threats to counter.
>>>
>>>So, I appreciate the Supplement(ASE).
>>>
>>>However, today, how could we deal with the current version CC Ver 2.1?
>>>
>>>Question is:
>>>1) Does the simplified requirements for the description of threats and
>>>security
>>>   objectives in the Supplement(ASE) mean a conceptual change
>>>   from the CC Ver 2.1?
>>>2) Or, is it just a refinement of the CC Ver 2.1 for the better and correct
>>>    interpretation?
>>>
>>>If '2)' is true,  we are happy.
>>>Because, I think, it would be possible to evaluate threats and security
>>>objectives
>>>  similarly, based on the Supplement(ASE).
>>>Am I right?
>>>
>>>If not, ST evaluation could be very hard under the CC Ver2.1.
>>>The ST evaluator needs to determine that:
>>>- any threats are described
>>>- the suitability of security objectives are demonstrated
>>>To determine the completeness of suitability is hard and may be subjective.
>>>
>>>Regards,
>>>    Yokota
>>**************************************************************************
>>* Opinions expressed are not intended to reflect an official position
>>**************************************************************************
>>* Gary Stoneburner
>>* Computer Security Division, National Institute of Standards & Technology
>>* 100 Bureau Drive, Stop 8930, Gaithersburg, MD 20877-8930
>>* Phone: 301-975-5394, FAX: 301-948-0279, Email: Stoneburner@nist.gov
>>**************************************************************************
>**************************************************************************
>* Opinions expressed are not intended to reflect an official position
>**************************************************************************
>* Gary Stoneburner
>* Computer Security Division, National Institute of Standards & Technology
>* 100 Bureau Drive, Stop 8930, Gaithersburg, MD 20877-8930
>* Phone: 301-975-5394, FAX: 301-948-0279, Email: Stoneburner@nist.gov
>**************************************************************************