Lessons Learned
Lessons shaped by engineering practice, project coordination, customer collaboration, technical communication, and continuous personal development.
From Experience to Framework
The lessons on this page have been generalized to preserve confidentiality while showing how practical experience shaped the Integrated Lifecycle Management Framework.
Each lesson follows a consistent structure:
Observed Condition
The recurring circumstance, gap, or behavior that affected engineering execution.
Lesson Learned
The broader technical, organizational, or interpersonal principle identified through experience.
Framework Response
The practice incorporated into Program Execution, Information Management, or Engineering Analysis to reduce the likelihood or impact of the condition recurring.
The lessons are intended to remain practical. They connect experience to actions that can improve future projects, strengthen knowledge transfer, support onboarding, and help multidisciplinary teams work with greater visibility, traceability, verifiability, and continuity.
1. Understanding the Customer Is Part of Engineering
Observed Condition
Customers and engineering teams may describe the same problem from different perspectives.
A customer may communicate an operational frustration, identify a visible symptom, or request a specific modification. The engineering team may then focus on implementing the requested solution without fully understanding the underlying need, operating environment, constraints, or broader system impact.
This can result in a technically correct response that does not fully address the customer’s actual pain point.
Communication challenges may also arise when technical information is presented at the wrong level. An explanation may be too detailed for a decision-maker, too simplified for the engineer responsible for implementation, or disconnected from the practical effect on the customer’s work.
As technology increasingly supports calculations, modeling, automation, data processing, and other technical activities, the ability to listen, understand context, build trust, and communicate clearly becomes even more important.
Lesson Learned
Effective engineering begins with understanding people.
Technical knowledge is essential, but it must be supported by the ability to listen carefully, ask meaningful questions, recognize the difference between a symptom and an underlying need, and translate that need into an actionable engineering problem.
Customer interaction is not separate from engineering. It is part of requirements development, system analysis, decision-making, implementation, and verification.
An engineer should be able to communicate at multiple levels:
The direct answer
What does the customer or stakeholder need to know?The practical meaning
Why does the issue or recommendation matter to their work, system, or mission?The technical basis
What requirement, evidence, analysis, model, or test supports the conclusion?The next action
What should happen as a result, who is responsible, and when is it expected?
The engineer should also be prepared to provide greater technical depth as questions arise without losing the clarity of the original explanation.
A customer’s requested solution should not automatically be treated as the requirement. The engineer must first understand the problem being experienced, the desired outcome, the constraints that shape an acceptable response, and the criteria that will demonstrate success.
When multiple solutions are possible, the customer should be given enough information to understand the relevant advantages, limitations, risks, costs, schedule impacts, and operational consequences.
A technically sound solution that does not address the customer’s actual need is not a complete solution.
Framework Response
Translate customer needs into clear requirements, communicate technical information at the appropriate level, document decisions and expectations, and verify that the implemented solution addresses the original concern.
Primary Framework Connection
All Three Frameworks
2. Visibility Drives Decision Velocity
Observed Condition
Project information may exist across schedules, emails, meeting notes, spreadsheets, technical documents, procurement records, models, and individual knowledge.
Although the information is technically available, it may still be difficult to determine:
What has been completed
What remains open
Who owns the next action
Which decisions are preventing progress
What information is still required
Which hardware, software, vendor, or approval dependency is affecting the schedule
What downstream work may be delayed
Whether the information being reviewed is current
This creates unnecessary effort before the actual technical review or decision can begin. Team members may spend more time locating, validating, and reconciling information than evaluating the issue itself.
The problem becomes more significant when different stakeholders need different levels of detail. A manager may need a concise understanding of status, risk, and required decisions, while an engineer may need the applicable requirement, interface, configuration, analysis, and verification evidence.
When these views are not connected, teams may develop different interpretations of the same project condition.
In one engineering-review process, reorganizing the information and clarifying the review path reduced typical turnaround time from approximately one to three days to one to three hours.
The technical rigor of the review did not change. The improvement came from making the relevant information easier to locate, understand, evaluate, and communicate.
Lesson Learned
Information creates value when it can be understood and acted upon.
Visibility allows teams to recognize issues earlier, direct questions to the correct people, identify bottlenecks, and make decisions with less unnecessary investigation.
This does not mean that every stakeholder should receive every technical detail. The information should be organized so that each person can access the level of detail appropriate to their role while remaining connected to the same authoritative source.
A useful project view should allow someone to quickly understand:
What is happening
Why it matters
Who owns the response
What is preventing progress
What happens next
When the issue or deliverable is expected to be resolved
Visibility also improves accountability. When owners, actions, dependencies, and completion dates are clearly presented, uncertainty becomes easier to distinguish from inactivity.
It does not eliminate technical complexity, but it reduces the additional complexity created by fragmented information and unclear communication.
Framework Response
Create connected execution and technical views that show current status, ownership, dependencies, risks, required decisions, next actions, and supporting evidence at the level appropriate to each audience.
Primary Framework Connection
Program Execution and Information Management
3. Complex Systems Become Manageable Through Decomposition
Observed Condition
Complex projects may be discussed as one large effort without clearly separating the systems, subsystems, components, software modules, interfaces, responsibilities, and lifecycle activities that make the work possible.
When the effort is viewed only at a high level, important details can become difficult to manage:
Ownership may remain unclear.
Interfaces may be overlooked.
Dependencies may remain hidden.
Technical issues may be assigned to the wrong discipline.
Procurement and installation impacts may not be identified early.
Testing and verification responsibilities may be incomplete.
Stakeholders may develop different interpretations of the system.
A local change may create effects elsewhere that are not immediately visible.
This becomes especially significant in interconnected electromechanical systems, where mechanical, electrical, software, controls, data, and human elements operate together.
A change that appears isolated may affect mechanical fit, electrical signaling, software behavior, control logic, procurement, installation, testing, configuration records, and customer operations.
Without decomposition, teams may understand individual tasks while still lacking a clear view of how the system functions as a whole.
Lesson Learned
Complexity becomes manageable when the system and the work are separated into understandable elements while preserving the relationships between them.
Decomposition allows the team to determine:
What the system contains
What each element is intended to do
How the elements interact
Who owns each area
What dependencies exist
How a change may propagate
How each element will be integrated and verified
The goal is not to divide the project into isolated disciplines. It is to create enough structure for each element to be understood individually while maintaining visibility into the integrated system.
This is particularly important in robotics and multidisciplinary engineering because a symptom may appear in one area while the actual cause originates somewhere else.
For example:
A mechanical tolerance issue may appear as unstable control behavior.
Electrical interference may appear as incorrect sensor data.
A software timing issue may appear as actuator inconsistency.
A configuration mismatch may appear as a component failure.
An undocumented interface change may affect installation, testing, and system performance.
Decomposition provides the structure needed to investigate these relationships without losing the system-level perspective.
Framework Response
Organize the system into identifiable functions, components, interfaces, owners, dependencies, and verification activities while preserving the relationships needed to understand system-wide impacts.
Primary Framework Connection
All Three Frameworks
4. Ownership Must Be Explicit
Observed Condition
Projects may identify the work that needs to be completed without clearly identifying who is responsible for completing, reviewing, approving, maintaining, or closing it.
An action may appear on a schedule or meeting agenda, but responsibility can remain unclear when:
Multiple organizations are involved
The task crosses engineering disciplines
A customer or vendor must provide input
One person performs the work while another holds approval authority
Ownership changes during the project
A deliverable depends on supporting information from several groups
The responsible person is assumed rather than formally identified
This creates uncertainty around who should act, who has the authority to decide, and who is responsible for confirming that the work is complete.
Unclear ownership can also create the appearance of progress without actual closure. A task may be discussed repeatedly, forwarded between groups, or shown as active for an extended period because no single person is accountable for moving it to the next state.
The same issue applies beyond schedule actions. Ownership may also be unclear for:
Requirements
Components
Interfaces
Software modules
Documents
Models
Assumptions
Technical decisions
Discrepancies
Test activities
Verification evidence
Configuration records
When ownership is not established, important information may become outdated, conflicting decisions may remain unresolved, and required work may be delayed until someone informally assumes responsibility.
Lesson Learned
A deliverable without a clear owner is a project risk.
Ownership creates a direct connection between the work and the person responsible for advancing it. It does not mean that one individual must perform every related activity. It means that someone must be accountable for coordinating the necessary contributors, maintaining visibility, escalating barriers, and confirming completion.
Responsibility and authority should also be distinguished.
The person completing the technical work may not be the same person who approves the result. A vendor may provide information without owning the final decision. A project lead may coordinate the action while a technical authority determines whether the solution is acceptable.
Clear ownership should therefore answer:
Who is responsible for completing or coordinating the work?
Who provides the required technical or organizational support?
Who has the authority to approve the result?
Who verifies that the work meets the applicable requirement or acceptance criterion?
Who maintains the resulting information after the immediate action is closed?
Ownership should remain visible when work is transferred between people or organizations. A change in personnel should not cause the action, rationale, or responsibility to disappear.
Explicit ownership also allows people to operate effectively within their roles. Engineers can focus on technical execution, project leads can coordinate dependencies and schedules, customers can clarify operational needs, vendors can provide contracted deliverables, and approval authorities can make decisions using the required evidence.
Framework Response
Assign a responsible owner, supporting contributors, approval authority, verification owner, and expected completion date to each significant deliverable, decision, interface, discrepancy, and information set.
Primary Framework Connection
Program Execution
5. Communication Must Be Closed-Loop
Observed Condition
Technical questions, decisions, and action items are often discussed across meetings, phone calls, instant messages, emails, and informal conversations.
A discussion may resolve the immediate issue, but the project can still remain exposed when the outcome is not documented clearly.
Common problems include:
Different participants leaving with different interpretations
Decisions being made without preserving the rationale
Actions being discussed without assigning an owner
Verbal agreements not being reflected in schedules, requirements, or technical records
People outside the original discussion remaining unaware of the outcome
Follow-up actions being assumed rather than confirmed
Changes being implemented in one area without notifying affected groups
Future engineers being unable to determine why a decision was made
The risk becomes greater when several disciplines, customers, vendors, or approval authorities are involved. A decision that appears clear within one group may have different implications for hardware, software, interfaces, procurement, installation, testing, or operations.
Communication can also fail when technical information is sent without making the required response clear. A message may contain detailed background but not identify:
What decision is needed
Who must respond
What action should occur
When the response is required
What happens if the issue is not resolved
In these situations, information has been transmitted, but communication has not been completed.
Lesson Learned
Communication is complete only when the relevant parties share the same understanding of the issue, decision, responsibilities, and next actions.
A practical communication sequence is:
Provide the issue and available context in writing.
Discuss the matter directly when clarification or rapid resolution is needed.
Confirm the decision, rationale, and affected areas.
Document the actions, owners, dependencies, and expected completion dates.
Distribute the record to the people responsible for acting on or being affected by the decision.
Follow up until the action is verified and closed.
This approach combines the speed of direct conversation with the continuity of a written project record.
Closed-loop communication does not mean documenting every conversation in excessive detail. The record should preserve the information needed for execution and future understanding:
What issue was raised
What information was considered
What was decided
Why the decision was made
Who owns the resulting actions
What systems, documents, or activities are affected
When the work is expected to be completed
What evidence will demonstrate closure
Communication should also be adapted to the audience.
A technical team may require detailed requirements, interfaces, assumptions, and analysis. A customer or manager may need a concise explanation of the issue, practical impact, available options, recommendation, and expected resolution timeline.
Both views should communicate the same underlying decision.
Closed-loop communication creates accountability, reduces misunderstanding, preserves engineering intent, and allows the project to continue without depending on the memory of the original participants.
Framework Response
Document significant questions, decisions, rationale, owners, affected areas, next actions, and completion dates, then follow through until the outcome is confirmed and closed.
Primary Framework Connection
Program Execution and Information Management
6. Tribal Knowledge Is an Organizational Risk
Observed Condition
Important engineering knowledge is often retained by individuals rather than preserved within the project record.
A team may know who to ask about a particular subsystem, interface, customer expectation, procurement issue, software behavior, or historical design decision. As long as that person remains available, the process may appear to work.
The risk becomes visible when:
The original engineer changes roles or leaves the organization
A new team member joins the project
Work is transferred between departments
A project resumes after a long pause
The same issue occurs on a later system
A customer or vendor asks why a past decision was made
A document shows the final result but not the reasoning behind it
A model, script, or codebase can only be maintained by its original author
In these situations, the information may technically exist, but the context required to understand and use it may be missing.
Tribal knowledge can include:
Why a design approach was selected
Which alternatives were rejected
Which assumptions were made
How an interface actually behaves
Which requirement caused a change
How a test should be interpreted
What temporary workaround remains in place
Which vendor or customer constraint shaped the solution
What limitations are known but undocumented
Which process steps are required for successful execution
The result is repeated investigation, slower onboarding, inconsistent decisions, and unnecessary dependence on specific individuals.
Lesson Learned
A process is not sustainable when its success depends on knowing who to ask.
Engineering knowledge should be preserved so that another person can understand the system, continue the work, and evaluate future changes without reconstructing the project from memory.
This does not mean documenting every minor detail. The goal is to preserve the information that gives the work meaning:
What was done
Why it was done
What information supported the decision
What assumptions or limitations remain
What systems and interfaces are affected
How the result was verified
What should be considered before making a future change
Knowledge transfer should also be iterative rather than postponed until project closeout.
Teams can preserve and distribute knowledge through:
Short technical summaries
Internal presentations
Technical learning sessions
Design and analysis reviews
Updated checklists and templates
Documented code and model repositories
Decision records
Lessons-learned discussions
Onboarding guides
Links between requirements, analyses, changes, and evidence
These methods allow one person’s experience to become reusable organizational knowledge.
Well-documented technical work also improves review quality. A reviewer should be able to understand the reasoning without relying entirely on the original author to explain it verbally.
The strongest form of knowledge transfer allows someone new to determine:
What the system or process is intended to do
What information is authoritative
What decisions shaped the current state
What assumptions and risks remain active
What must be considered before continuing or changing the work
Framework Response
Preserve the purpose, rationale, assumptions, decisions, interfaces, configuration, methods, and verification evidence needed for another engineer to understand and continue the work.
Primary Framework Connection
Information Management
7. Traceability Preserves Engineering Intent
Observed Condition
Engineering work often progresses through multiple revisions, decisions, reviews, and changes.
A drawing may reach Revision F, a software build may move through several releases, or a requirement may be rewritten multiple times. The final version may show what exists now without explaining how the project arrived there.
This creates difficulty when:
A future engineer needs to understand why a design changed
A customer asks which requirement drove the modification
A test result no longer matches the current configuration
A discrepancy appears after several revisions
A software change affects an interface or verification activity
Multiple documents are updated at different times
An approval is recorded without preserving the supporting rationale
A change is implemented but not reflected across all affected records
Revision history alone may show that something changed, but not:
What initiated the change
Why the change was necessary
Which alternatives were considered
What assumptions were made
What systems and interfaces were affected
Who approved the decision
How the updated solution was verified
When these relationships are missing, future work may repeat earlier investigations, reverse a valid decision, or introduce changes without understanding the original engineering intent.
Traceability is especially important across interconnected systems because a single change may affect requirements, hardware, software, interfaces, procurement, installation, testing, and documentation simultaneously.
Lesson Learned
Engineering information should show not only the current result, but the reasoning and evidence that produced it.
Traceability preserves engineering intent by connecting:
Need → Requirement → Design → Implementation → Verification Evidence
It should also allow a reviewer to move in the opposite direction:
Verification Evidence → Implemented Element → Requirement → Original Need
This connection helps determine whether the current system still satisfies its intended purpose and whether a proposed change affects more than the immediately visible component.
Traceability should answer:
What need or issue initiated the work?
What requirement or decision defined the expected outcome?
What design or implementation addressed it?
What changed during development?
Why was the selected approach approved?
What evidence demonstrates that the result is acceptable?
A document at Revision F should contain enough history for another engineer to understand why it reached Revision F.
The same principle applies to code, models, interface definitions, test procedures, and configuration records. The artifact should remain connected to the requirement, decision, change, and evidence that give it meaning.
Traceability does not require every relationship to be recreated manually within one document. The information may exist across several controlled tools or repositories, provided that the relationships can be followed reliably.
Framework Response
Connect requirements, decisions, designs, changes, implementations, configurations, and verification evidence so that the current system state and the reasoning behind it can be understood.
Primary Framework Connection
Information Management
8. Schedule Risk Is Driven by Dependencies, Not Dates Alone
Observed Condition
Project schedules often show planned start dates, completion dates, and major milestones without fully identifying what must be available before each activity can begin.
A task may appear to be on schedule even though it depends on:
A long-lead hardware component
A software release
A vendor drawing or technical submittal
A customer decision
An approved interface definition
A completed analysis
A contract or purchase order
Manufacturing capacity
Installation access
Test equipment
Configuration approval
Objective quality evidence
When these dependencies are not visible, the schedule can create a false sense of confidence.
The risk may not become apparent until a downstream activity is ready to begin and the required material, information, approval, or technical output is unavailable.
This is especially important for complex systems because the most schedule-critical item may not be the largest or most technically complicated component. A relatively small part, interface decision, software dependency, or approval can prevent installation, integration, testing, or verification from progressing.
Material status can also be misleading when it is tracked only as ordered or received. The team may still need to determine whether the item:
Matches the approved configuration
Has completed inspection
Includes the required documentation
Is available at the correct location
Can be installed with the current design
Supports the planned test activity
Requires additional vendor or engineering action
Similarly, software may be described as complete while still requiring integration, configuration, defect resolution, release approval, deployment, or regression testing.
A schedule that focuses only on dates may therefore show activity without revealing whether the project is truly ready to advance.
Lesson Learned
Schedule performance is controlled by dependencies, not dates alone.
A meaningful schedule should show the relationships between engineering decisions, hardware, software, procurement, manufacturing, installation, integration, testing, and approval activities.
The most important question is not only:
When is this task expected to finish?
It is also:
What must be completed, delivered, approved, or verified before the next activity can begin?
A component or activity becomes schedule-critical when its delay prevents another required activity from starting, continuing, or completing.
This means schedule risk should be evaluated by considering:
What the activity depends on
What downstream work depends on it
How much schedule margin remains
When mitigation must begin
Whether an alternate path exists
When the issue should be escalated
Dependencies should also be evaluated across disciplines.
A hardware delay may affect software integration. A missing interface definition may delay both design and testing. A customer decision may prevent procurement. A software defect may delay system acceptance even when all physical installation work is complete.
The schedule should therefore provide a progressive view of how the project is moving toward integrated readiness, rather than functioning as a list of isolated completion dates.
Framework Response
Track critical dependencies, responsible owners, need dates, current forecasts, downstream impacts, available schedule margin, mitigation actions, and escalation points.
Primary Framework Connection
Program Execution
9. Procurement, Manufacturing, and Development Are Engineering Activities
Observed Condition
Procurement, manufacturing, and software development are sometimes treated as supporting administrative activities rather than as part of the engineering process.
A design may appear technically complete while the project still depends on:
Vendor availability
Long-lead components
Obsolete or discontinued parts
Manufacturing capacity
Specialized materials or processes
Contract placement
Technical proposal approval
Software development
Firmware release
Inspection and quality documentation
Shipping and receipt
Installation readiness
Customer- or government-furnished equipment
When these activities are not connected to the engineering plan, technical and schedule risks may remain hidden until they begin affecting installation, integration, testing, or delivery.
A selected component may satisfy the immediate design requirement but still create lifecycle problems if it:
Cannot be sourced reliably
Has an unacceptable lead time
Requires a manufacturing process that is not available
Does not include the necessary technical documentation
Is difficult to inspect or verify
Creates future obsolescence risk
Requires an interface change
Cannot be maintained or replaced efficiently
Depends on a vendor-controlled configuration
The same principle applies to software.
A capability may be described as developed while still requiring:
Integration with the target hardware
Interface validation
Configuration management
Defect resolution
Release approval
Deployment
Cybersecurity review
Regression testing
User acceptance
Supporting documentation
Nontraditional manufacturing approaches may also require additional engineering consideration. A temporary fixture, mock-up, alternate manufacturing method, or 3D-printed component may help resolve a schedule or availability issue, but the approach must still be evaluated against the applicable requirements, material limitations, interfaces, safety concerns, configuration controls, and verification needs.
Treating these activities as separate from engineering can result in a system that is technically designed but not realistically producible, procurable, integrable, testable, or supportable.
Lesson Learned
A design is not complete simply because it works in theory.
Engineering decisions must consider how the solution will be sourced, manufactured, developed, inspected, installed, integrated, verified, maintained, and replaced throughout its lifecycle.
Procurement should communicate more than whether an item was ordered. The engineering team should understand:
Whether the correct configuration was specified
Whether the vendor can meet the technical requirements
Whether the proposed lead time supports the schedule
Whether alternate sources or methods exist
Whether inspection or objective quality evidence is required
Whether the item affects downstream installation or testing
Whether obsolescence or lifecycle support creates additional risk
Vendor communication is therefore part of technical execution. Engineers may need to clarify requirements, evaluate proposals, review drawings, resolve discrepancies, assess alternatives, and confirm that the delivered product matches the intended design.
Manufacturing decisions also influence tolerances, materials, interfaces, inspection methods, cost, schedule, and system performance. A design that cannot be manufactured consistently or verified adequately may require modification even when the underlying technical concept is valid.
Software and firmware development should be evaluated with the same lifecycle awareness. The engineering question is not only whether the code performs the intended function, but whether it can be integrated, configured, tested, released, maintained, and reproduced within the complete system.
The most effective project teams connect these activities early enough to identify risks before they become schedule-critical.
Framework Response
Integrate procurement, manufacturing, vendor coordination, software development, configuration, inspection, and release status into technical planning and lifecycle decision-making.
Primary Framework Connection
Program Execution and Information Management
10. Interfaces Are Common Sources of Hidden Risk
Observed Condition
Technical issues are often first noticed within a specific component, discipline, or phase of work, even though the underlying cause exists at an interface.
An apparent hardware problem may be caused by software timing. A software issue may originate from an incorrect sensor signal. An installation discrepancy may result from an outdated model, drawing, or interface definition. A customer concern may reveal that two groups are working from different assumptions about how connected systems are expected to interact.
Interface risks may involve:
Mechanical fit and alignment
Electrical power, grounding, or signal compatibility
Software APIs and communication protocols
Data formats, units, and expected ranges
Timing, latency, and synchronization
Configuration differences
Environmental conditions
Installation boundaries
Customer- or vendor-provided equipment
Organizational handoffs and approval responsibilities
These issues can remain hidden because each group may correctly complete its individual work while the combined system still fails to operate as intended.
In one generalized example, a design concern raised during customer coordination could not be confirmed using the available project documentation. Comparing the customer-provided information with the current interface model revealed that a model update had changed the expected configuration, but the change had not been communicated consistently across all affected groups.
The individual records appeared reasonable when reviewed separately. The discrepancy became visible only when the connected information was compared.
Interface problems are also difficult to detect when:
Different groups use different revisions
Ownership is divided between organizations
One party assumes another party will verify compatibility
Requirements describe individual components but not their interactions
Changes are communicated within one discipline but not across the system
Integration testing occurs too late to influence the design efficiently
Interface assumptions remain undocumented
The result may be rework, delayed installation, failed testing, incompatible components, or disagreement about which group is responsible for correcting the issue.
Lesson Learned
A system can contain individually correct components and still fail because the connections between them were not clearly defined, controlled, communicated, or verified.
Interfaces should therefore be treated as engineering elements in their own right.
Each significant interface should have an understood purpose, definition, owner, applicable configuration, and method of verification. The connected parties should agree on what crosses the boundary, under what conditions, and how successful interaction will be demonstrated.
Interface management requires more than documenting connection points. It also requires understanding:
What each side expects from the other
Which requirements govern the interaction
Which assumptions are being made
Which organization owns the definition
Which configuration is authoritative
What changes may affect the connected system
How compatibility will be tested
Who must be notified when the interface changes
This is especially important in multidisciplinary systems because a change may be technically valid within one area while still creating an unintended effect elsewhere.
A software update may change timing. A mechanical redesign may affect sensor placement. A substituted electrical component may alter communication behavior. A customer-provided configuration may not match the integration assumptions used by the engineering team.
The interface is where those individual decisions become system behavior.
Interface verification should also occur before final integration whenever practical. Early use of models, interface reviews, mock-ups, simulations, test fixtures, stubs, emulators, or controlled demonstrations can expose incompatibilities while there is still time to correct them efficiently.
Framework Response
Define interface ownership, requirements, assumptions, configurations, affected parties, change impacts, and verification methods, then review them whenever a connected element changes.
Primary Framework Connection
Information Management and Engineering Analysis
11. Data Discrepancies Must Be Explained, Not Ignored
Observed Condition
Engineering decisions often rely on information collected from multiple sources:
Physical measurements
Test results
Software logs
Simulation outputs
Inspection records
Customer observations
Vendor data
Drawings and models
Historical records
Configuration databases
These sources do not always agree.
A simulation may predict one result while the physical test produces another. Two sensors may report different values. A drawing may show a configuration that does not match the installed system. Software logs may indicate an event sequence that conflicts with witness observations or test instrumentation.
When results disagree, there may be pressure to select the value that appears most reasonable, average the data, remove the outlier, or continue using the source that best supports the expected conclusion.
That can hide important information.
A discrepancy may result from:
Different hardware or software configurations
Incorrect units or reference frames
Sensor calibration error
Measurement uncertainty
Timing or synchronization differences
Sampling-rate differences
Data filtering or preprocessing
Environmental or operating-condition changes
Incorrect model assumptions
Outdated drawings or documentation
Instrumentation limitations
Interface behavior
Test setup variation
Data-entry or transcription error
An actual change in system behavior
The disagreement itself may be evidence of a configuration, measurement, interface, or system problem.
Data can also appear consistent while still being misleading. Similar values may have been collected under different operating conditions, generated using the same incorrect assumption, or copied from a common outdated source.
For that reason, agreement between records does not automatically establish validity, just as disagreement does not automatically identify which source is incorrect.
Lesson Learned
Conflicting data should be investigated, not silently corrected or removed.
The purpose of analysis is not to force all information into agreement. It is to understand why the information differs and determine what conclusion the available evidence can reasonably support.
A data discrepancy should prompt questions such as:
Are the data sets associated with the same system configuration?
Were they collected under the same operating conditions?
Are the units, reference frames, and timestamps consistent?
Were the instruments calibrated and operating correctly?
Was the data filtered, converted, or processed differently?
Does the model represent the actual physical system?
Could the difference result from an interface, timing, or synchronization issue?
Is one source outdated, incomplete, or outside its intended range of use?
Could the discrepancy represent real system variability?
What additional evidence would distinguish between the possible explanations?
The explanation should be proportional to the evidence.
When the cause cannot be confirmed, the analysis should clearly state:
What is known
What remains unknown
What explanation is currently considered most likely
What evidence supports that explanation
What evidence contradicts it
What alternative explanations remain possible
What level of confidence is appropriate
What action is required next
An unresolved discrepancy is not automatically a failed analysis. Clearly identifying uncertainty and defining the next step is more valuable than presenting an unsupported conclusion.
Visuals can be especially useful when communicating discrepancies. Before-and-after plots, test-versus-model overlays, time-series graphs, scatter plots, configuration comparisons, and uncertainty ranges can help reviewers see where the results agree and where they diverge.
The visual should identify the applicable configuration, conditions, units, limits, and intended conclusion so the reviewer is not required to infer the meaning from raw data
Framework Response
Document conflicting results, evaluate the most plausible causes, state the level of confidence, and identify the additional evidence or actions required to resolve the discrepancy.
Primary Framework Connection
Engineering Analysis
12. Recommendations Require Option Analysis
Observed Condition
Engineering teams may identify a technically feasible solution and move quickly toward implementation without fully comparing it against other available approaches.
The first solution proposed may appear reasonable because it:
Resolves the immediate technical issue
Uses familiar tools or methods
Can be implemented quickly
Aligns with the preference of a particular stakeholder
Avoids reopening an earlier design decision
Requires less initial explanation
However, a technically workable option may still create problems elsewhere.
It may:
Introduce new interface risks
Increase lifecycle cost
Require difficult procurement
Create software or hardware dependencies
Reduce maintainability
Add schedule risk
Require extensive regression testing
Limit future upgrades
Shift risk to another subsystem
Address the symptom without resolving the underlying cause
In some cases, several options may be presented without using the same criteria to evaluate each one. One alternative may be described primarily by cost, another by technical performance, and another by schedule. Without a consistent comparison, the recommendation can become difficult to defend.
There is also a risk of treating option analysis as a formality after a preferred solution has already been selected. This can cause the evaluation to justify an existing preference rather than objectively compare the available approaches.
Lesson Learned
A recommendation is stronger when it demonstrates why the selected option is preferable to the realistic alternatives.
When multiple viable approaches exist, each option should be evaluated using criteria appropriate to the decision.
These may include:
Requirements compliance
Safety
Technical performance
Reliability
Maintainability
Interface compatibility
Cost
Schedule
Procurement availability
Manufacturing feasibility
Software-development effort
Installation impact
Verification complexity
Operational impact
Residual risk
Reversibility
Long-term lifecycle effects
The purpose of option analysis is not always to identify a universally superior solution. Engineering decisions often involve tradeoffs.
One option may provide the strongest technical performance but require more time and cost. Another may be easier to implement but create greater maintenance burden. A temporary mitigation may be acceptable for immediate containment while a more complete solution is developed.
The recommendation should therefore explain:
What options were considered
What criteria were used
What assumptions influenced the comparison
What advantages and disadvantages apply to each option
What risks and dependencies remain
Why the selected option best supports the current need
Under what conditions the recommendation should be reconsidered
The level of analysis should match the significance of the decision.
A minor and reversible change may require only a concise comparison. A safety-critical, expensive, difficult-to-reverse, or system-wide decision may require a formal trade study, sensitivity analysis, modeling, testing, or stakeholder review.
The evaluation should also recognize when “no change” is a valid option. Maintaining the current configuration may be appropriate when the issue does not justify the cost, risk, or disruption associated with modification, provided that the decision and residual risk are clearly documented.
A recommendation should not simply state what should be done. It should provide enough evidence for the decision-maker to understand why that action is appropriate.
Framework Response
Compare viable options using consistent criteria, document the tradeoffs and assumptions, and explain why the recommended approach best supports the technical and lifecycle objectives.
Primary Framework Connection
Engineering Analysis
13. Verification Begins Before Testing
Observed Condition
Verification is sometimes treated as an activity that begins after design, development, manufacturing, or installation is complete.
The system may reach a formal test event before the team has fully agreed on:
Which requirement is being verified
What correct performance looks like
Which configuration should be tested
What data must be collected
What equipment or facilities are required
Who has authority to accept the result
What evidence is needed for closure
How failures or unexpected results will be handled
This can lead to testing that produces data without producing a clear engineering decision.
A test may be completed successfully from an operational perspective while still failing to demonstrate compliance with the applicable requirement. Conversely, a result may appear unsuccessful because the acceptance criteria, operating conditions, or expected behavior were not clearly defined before the test began.
Verification can also be delayed when required items are identified too late, such as:
Test equipment
Instrumentation
Approved procedures
Software builds
Hardware configurations
Calibration records
Customer or approval-authority participation
Environmental conditions
Objective quality evidence
Data-storage and reporting methods
In interconnected systems, verifying individual components may not be sufficient. Mechanical, electrical, software, control, and interface behavior may each appear acceptable independently while the integrated system still fails to perform as intended.
Late verification planning can expose problems after procurement, manufacturing, installation, or software development has already progressed far enough that changes become more difficult and expensive.
Lesson Learned
Verification should influence the engineering approach from the beginning, not only evaluate the result at the end.
When a requirement is created or a solution is proposed, the team should also consider:
How will this be demonstrated?
What evidence will be considered acceptable?
What configuration and conditions must exist?
Who will review and approve the result?
What resources must be available?
What related functions or interfaces could be affected?
Thinking about verification early can reveal whether a requirement is measurable, whether the design can be tested, and whether the necessary evidence can realistically be produced.
Verification planning can also expose ambiguity.
A requirement that cannot be tied to a clear inspection, analysis, demonstration, or test method may require refinement before design work continues.
The verification method should match the nature of the requirement. Depending on the system, compliance may be demonstrated through:
Inspection
Analysis
Demonstration
Testing
Simulation
Model correlation
Document review
A combination of methods
The team should also distinguish between component verification and system-level validation.
A component may satisfy its allocated requirement while the integrated system still fails to support the customer’s intended use. Verification asks whether the system was built correctly against the defined requirements. Validation asks whether the resulting system addresses the actual need.
Both should be considered before the project reaches final acceptance.
Early verification planning also improves schedule and procurement decisions. It allows the team to identify required equipment, facilities, software environments, fixtures, personnel, approvals, and evidence before they become late project constraints.
Framework Response
Define the verification method, acceptance criteria, configuration, resources, evidence, and approval authority while requirements and solutions are being developed.
Primary Framework Connection
Program Execution and Engineering Analysis
14. Task Completion Does Not Equal System Readiness
Observed Condition
Projects may report individual tasks, components, or deliverables as complete even though the integrated system is not ready to proceed.
A drawing may be approved, hardware may be installed, software may be released, or a test procedure may be written. Each activity may be complete within its own area while unresolved conditions still prevent the broader system from moving forward.
Common examples include:
Hardware is installed but has not been inspected or configured.
Software is released but has not been integrated with the target hardware.
A component has been received but does not match the approved configuration.
A test procedure exists but the required equipment or facilities are unavailable.
Individual interfaces were reviewed but end-to-end behavior has not been demonstrated.
Documentation is complete but does not reflect the installed system.
A discrepancy is technically understood but has not been formally dispositioned.
Required customer, safety, quality, or approval authorities have not agreed that entry criteria are satisfied.
Supporting personnel, vendors, or external systems are unavailable for the planned event.
Individual requirements have evidence, but the complete operational use case has not been validated.
When completion is measured only by isolated task status, the project may appear more mature than it actually is.
This can create pressure to begin installation, integration, testing, demonstration, or delivery before the necessary conditions are in place. The result may be avoidable delays, repeated work, failed test events, configuration confusion, or increased risk.
The issue is especially important in multidisciplinary systems because readiness depends on the combined state of hardware, software, interfaces, documentation, personnel, facilities, and verification evidence.
Lesson Learned
Completion and readiness are related, but they are not the same.
A completed task confirms that a defined activity reached its stated endpoint. Readiness confirms that the connected system, people, information, resources, and approvals are prepared for the next lifecycle event.
A project should therefore distinguish between:
Task completion — Has the individual activity been performed?
Deliverable acceptance — Does the output meet its defined criteria?
Integration readiness — Can the output be combined with the other system elements?
Test readiness — Are the system configuration, procedures, equipment, personnel, and acceptance criteria prepared?
Operational readiness — Can the system perform its intended function within the actual use environment?
Decision readiness — Is sufficient evidence available for the responsible authority to proceed?
Readiness should be evaluated against explicit entry criteria rather than assumed from a percentage-complete value or a collection of closed actions.
Before advancing, the team should be able to answer:
Are the applicable requirements and interfaces sufficiently mature?
Are the correct hardware and software configurations available?
Are open discrepancies understood and acceptably dispositioned?
Are required documents, procedures, facilities, equipment, and personnel available?
Are risks and limitations visible to the responsible decision-makers?
Is the required evidence available to support the decision?
Do the responsible technical and approval authorities agree that the entry criteria have been met?
A project does not need every issue to be eliminated before proceeding. It does need the remaining issues to be visible, understood, assigned, and evaluated against the risk of moving forward.
Readiness is therefore an evidence-based decision about whether the integrated effort can successfully enter the next phase.
Framework Response
Define measurable readiness criteria across hardware, software, interfaces, documentation, resources, open risks, verification evidence, and approval authority before advancing to the next lifecycle event.
Primary Framework Connection
Program Execution
15. Automation Should Support Judgment, Not Replace It
Observed Condition
Engineering teams often spend significant time performing repetitive information tasks that are necessary but do not always require sustained technical judgment.
These activities may include:
Searching for part information
Comparing revisions
Updating status fields
Consolidating action items
Checking naming conventions
Transferring information between tools
Generating recurring reports
Identifying missing records
Repeating standard calculations
Comparing configurations
Retrieving price, availability, or lead-time information
When these activities are completed manually, engineers may spend more time organizing and locating information than interpreting it.
Manual processes can also introduce:
Data-entry errors
Inconsistent formatting
Duplicated information
Missed updates
Delayed reporting
Differences between supposedly identical records
Dependence on individuals who understand the process
Limited time for reviewing unusual conditions or technical exceptions
Automation can reduce this burden, but it introduces a different risk when the output is accepted without understanding how it was produced.
A script, model, workflow, or artificial-intelligence tool may generate a result quickly while still relying on:
Incomplete inputs
Incorrect assumptions
Outdated data
Mismatched configurations
Hidden transformations
Inappropriate acceptance criteria
Limited operating ranges
Unverified source information
An automated result may appear more authoritative because it is produced consistently or presented professionally, even when the underlying logic or data is incorrect.
The same concern applies when automation removes the opportunity for engineers to notice unusual patterns. A system designed only to process expected cases may overlook the exception that reveals the actual engineering problem.
Lesson Learned
Automation is most valuable when it reduces repetitive effort and gives people more time to apply judgment.
The purpose of automation should not be to remove engineers from the process. It should allow them to focus on activities that require interpretation, communication, evaluation, and decision-making.
Appropriate uses may include:
Collecting and organizing data
Performing repeatable calculations
Generating standard visualizations
Identifying missing information
Comparing records or configurations
Flagging values outside expected limits
Updating routine status information
Producing draft reports
Maintaining links between related information
Human judgment remains necessary for determining:
Whether the input data is valid
Whether the assumptions are appropriate
Whether the method applies to the current system and configuration
Whether the result is technically reasonable
Whether unusual conditions require additional investigation
Whether the recommendation addresses the actual customer or operational need
Whether the risk is acceptable
Whether sufficient evidence exists for approval
Automation should make its inputs, assumptions, logic, and limitations visible enough for another person to review.
A result that cannot be explained, traced to its source, or reproduced should not be treated as authoritative solely because it was generated automatically.
The level of human review should also reflect the consequence of the decision.
A low-risk administrative update may require minimal review. A safety-related, mission-critical, expensive, or difficult-to-reverse engineering decision requires greater technical oversight and independent verification.
As technical tools continue to advance, the engineer’s role increasingly includes asking the right questions, recognizing limitations, interpreting outputs, communicating uncertainty, and determining when the result does not make sense.
Automation can accelerate engineering work. Judgment determines whether that work should be trusted.
Framework Response
Automate repetitive and traceable activities while preserving human review of data validity, assumptions, technical conclusions, risks, recommendations, and final approval.
Primary Framework Connection
All Three Frameworks
16. A Perfect Toolchain Is Not Required to Improve Engineering Management
Observed Condition
Engineering organizations do not always operate within one fully connected digital environment.
Requirements may be maintained in one system, schedules in another, technical documents in shared folders, software in version-controlled repositories, procurement information in business systems, and test evidence in separate databases or spreadsheets.
This is particularly common in established organizations where:
Legacy tools remain essential to current operations
Different departments adopted tools independently
Information systems were created at different points in time
Contractual or security requirements restrict integration
Employees use local processes to fill gaps between official systems
Historical records cannot be migrated easily
New digital-engineering capabilities are being introduced gradually
Full MBSE implementation requires time, funding, training, and organizational support
In these environments, project information may exist without forming a complete digital thread.
An engineer may need to move manually between:
Requirements
Schedules
Drawings and specifications
System models
Hardware records
Software repositories
Interface documentation
Procurement status
Analysis results
Test procedures
Verification evidence
Change and configuration records
The absence of a fully integrated toolchain can make it difficult to determine how the information relates, which source is authoritative, or what must be updated when a change occurs.
There can also be a tendency to delay process improvement until a larger software implementation, database migration, or MBSE initiative is completed.
This creates a risk that known problems continue because the ideal technical solution is not yet available.
The opposite problem can occur as well. An organization may purchase an advanced tool without first establishing:
Consistent naming
Information ownership
Configuration control
Clear workflows
Requirements discipline
Interface definitions
Review responsibilities
Expected outputs
User adoption
In that case, the organization gains a new application without necessarily improving how engineering information is created, connected, reviewed, or maintained.
Lesson Learned
A fully integrated toolchain can improve traceability and impact analysis, but meaningful process improvement does not have to wait for the perfect digital environment.
Teams can begin creating greater visibility, traceability, verifiability, and continuity by applying disciplined information practices within the tools already available.
Practical improvements may include:
Establishing authoritative sources
Using consistent system and component identifiers
Standardizing document and revision naming
Linking related records
Identifying owners
Preserving decision rationale
Maintaining interface and dependency matrices
Connecting requirements to verification evidence
Creating shared execution views
Documenting where information is located and how it should be used
A structured spreadsheet, controlled document index, linked repository, or shared project page can provide immediate value when it clearly connects the information needed to manage the work.
These methods do not replace MBSE. They can establish the information discipline and organizational habits needed for MBSE to succeed.
Model-Based Systems Engineering is most valuable when it makes relationships explicit across:
Stakeholder needs
Requirements
Functions
Behaviors
Components
Interfaces
Decisions
Changes
Verification activities
However, no model is perfect.
A model is an organized representation of the system based on the available information, selected scope, assumptions, and intended use. It may be incomplete, outdated, incorrectly connected, or too detailed in some areas and insufficiently detailed in others.
The model should therefore support engineering judgment rather than create the assumption that every relevant condition has already been represented.
The same principle applies to any tool.
A tool can improve consistency and visibility, but it cannot independently determine:
Which information is important
Whether the information is accurate
Whether the relationships are correct
Whether the process reflects how the work is actually performed
Whether users understand and maintain the system
Whether the resulting information supports better decisions
Process maturity and tool maturity should develop together.
Organizations can improve incrementally by first making the work understandable and repeatable, then increasing automation and integration where those improvements provide clear value.
The goal is not to create the most technically sophisticated environment. The goal is to create an engineering process in which information can be found, understood, connected, reviewed, updated, and used throughout the lifecycle of the system.
Framework Response
Use consistent identifiers, controlled records, visible ownership, and traceable relationships within the available tools, then introduce greater automation, MBSE, and toolchain integration as organizational maturity allows.
Primary Framework Connection
Information Management