에이전트 루프 진단과 문서 golden 회귀를 마감한다
- AgentLoopService의 컨텍스트 압축 완료 메시지와 query-view 요약 문자열 조립을 AgentLoopDiagnosticsFormatter로 분리해 루프 본체의 책임을 더 줄였다. - ChatStorageService 로드 경로에서 legacy .axchat의 누락된 tool_result preview를 synthetic preview로 즉시 복원하도록 보강했다. - HTML/DOCX golden 회귀와 legacy 저장 경로 회귀 테스트를 추가해 PPTX/XLSX에 이어 문서 품질 고정 범위를 확대했다. - 검증: dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\verify_loop_storage_golden\ -p:IntermediateOutputPath=obj\verify_loop_storage_golden\ (경고 0, 오류 0) - 검증: dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter AgentLoopDiagnosticsFormatterTests|ChatStorageServiceTests|HtmlSkillGoldenReportTests|DocxSkillGoldenDocumentTests|AgentMessageInvariantHelperTests|PptxSkillGoldenDeckTests|ExcelSkillGoldenWorkbookTests -p:OutputPath=bin\verify_loop_storage_golden_tests\ -p:IntermediateOutputPath=obj\verify_loop_storage_golden_tests\ (통과 10)
This commit is contained in:
@@ -1932,3 +1932,11 @@ MIT License
|
|||||||
- 문서 golden 회귀도 확대했습니다. [ExcelSkillGoldenWorkbookTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ExcelSkillGoldenWorkbookTests.cs)는 summary/dashboard/detail 구조와 formula, data validation, conditional formatting이 모두 들어간 운영 리뷰 workbook이 `Needs work: none`과 `Repair guide: none`을 유지하는지 검증합니다.
|
- 문서 golden 회귀도 확대했습니다. [ExcelSkillGoldenWorkbookTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ExcelSkillGoldenWorkbookTests.cs)는 summary/dashboard/detail 구조와 formula, data validation, conditional formatting이 모두 들어간 운영 리뷰 workbook이 `Needs work: none`과 `Repair guide: none`을 유지하는지 검증합니다.
|
||||||
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\` 경고 0 / 오류 0
|
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\` 경고 0 / 오류 0
|
||||||
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\` 통과 10
|
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\` 통과 10
|
||||||
|
|
||||||
|
업데이트: 2026-04-15 09:20 (KST)
|
||||||
|
- [AgentLoopDiagnosticsFormatter.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentLoopDiagnosticsFormatter.cs)를 추가해 [AgentLoopService.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentLoopService.cs)의 컨텍스트 압축 완료 메시지와 query-view 진단 문자열 조립을 별도 helper로 분리했습니다. 루프 본체는 orchestration에 더 가깝게 남기고, 진단 포맷은 테스트 가능한 단위로 떼어낸 변경입니다.
|
||||||
|
- [ChatStorageService.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/ChatStorageService.cs)는 로드 시점에도 `tool_result` preview 보정을 수행하도록 확장했습니다. 이로써 예전 버전에서 저장되어 preview가 비어 있는 legacy 대화도 재열기 순간 즉시 synthetic preview를 복원합니다.
|
||||||
|
- golden 회귀를 HTML/DOCX까지 확대했습니다. 새 [HtmlSkillGoldenReportTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/HtmlSkillGoldenReportTests.cs)는 board-grade HTML 보고서가 `Needs work: none`, `Repair guide: none`을 유지하는지 검증하고, 새 [DocxSkillGoldenDocumentTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/DocxSkillGoldenDocumentTests.cs)는 cover/TOC/template/header-footer를 갖춘 business DOCX 조립 결과가 품질 리뷰 `이슈 0`을 유지하는지 고정합니다.
|
||||||
|
- 저장 경로 회귀도 추가했습니다. 새 [ChatStorageServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ChatStorageServiceTests.cs)는 legacy `.axchat` 파일을 직접 만든 뒤 로드시 synthetic preview가 실제로 채워지는지 검증합니다.
|
||||||
|
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_loop_storage_golden\\ -p:IntermediateOutputPath=obj\\verify_loop_storage_golden\\` 경고 0 / 오류 0
|
||||||
|
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentLoopDiagnosticsFormatterTests|ChatStorageServiceTests|HtmlSkillGoldenReportTests|DocxSkillGoldenDocumentTests|AgentMessageInvariantHelperTests|PptxSkillGoldenDeckTests|ExcelSkillGoldenWorkbookTests" -p:OutputPath=bin\\verify_loop_storage_golden_tests\\ -p:IntermediateOutputPath=obj\\verify_loop_storage_golden_tests\\` 통과 10
|
||||||
|
|||||||
@@ -1003,3 +1003,11 @@ UI ?붿옄???洹쒕え 由ы뙥?좊쭅 ???꾪뿕 ?묒뾽 ??湲곕줉???덉쟾
|
|||||||
- 문서 golden 회귀도 한 단계 더 올렸습니다. 새 [ExcelSkillGoldenWorkbookTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ExcelSkillGoldenWorkbookTests.cs)는 summary/dashboard/detail 구조와 formula, data validation, conditional formatting이 모두 포함된 운영 리뷰 workbook이 `Needs work: none`, `Repair guide: none`을 유지하는지 확인합니다.
|
- 문서 golden 회귀도 한 단계 더 올렸습니다. 새 [ExcelSkillGoldenWorkbookTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ExcelSkillGoldenWorkbookTests.cs)는 summary/dashboard/detail 구조와 formula, data validation, conditional formatting이 모두 포함된 운영 리뷰 workbook이 `Needs work: none`, `Repair guide: none`을 유지하는지 확인합니다.
|
||||||
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\` 경고 0 / 오류 0
|
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\` 경고 0 / 오류 0
|
||||||
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\` 통과 10
|
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\` 통과 10
|
||||||
|
|
||||||
|
업데이트: 2026-04-15 09:20 (KST)
|
||||||
|
- 루프 진단 포맷을 분리했습니다. 새 [AgentLoopDiagnosticsFormatter.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentLoopDiagnosticsFormatter.cs)는 컨텍스트 압축 완료 이벤트와 query-view 요약 문자열을 전담하며, [AgentLoopService.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentLoopService.cs)는 해당 formatter를 호출하는 orchestration 역할로 더 가벼워졌습니다.
|
||||||
|
- 저장/재개 경로의 legacy preview 복원도 보강했습니다. [ChatStorageService.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/ChatStorageService.cs)는 복호화 직후 `PopulateMissingToolResultPreviews()`를 수행해 예전 저장본에서도 synthetic preview를 즉시 채웁니다.
|
||||||
|
- 테스트는 새 [AgentLoopDiagnosticsFormatterTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentLoopDiagnosticsFormatterTests.cs), [ChatStorageServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ChatStorageServiceTests.cs), [HtmlSkillGoldenReportTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/HtmlSkillGoldenReportTests.cs), [DocxSkillGoldenDocumentTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/DocxSkillGoldenDocumentTests.cs)를 추가해 루프 진단 문자열, legacy `.axchat` 복원, HTML/DOCX golden 품질을 회귀 고정했습니다.
|
||||||
|
- golden 범위는 이제 `PPTX + XLSX + HTML + DOCX`까지 확장되었습니다. HTML golden은 board-grade 보고서의 print frame/evidence/decision 구성을, DOCX golden은 template/TOC/header-footer/appendix가 포함된 business pack 조립을 기준 fixture로 삼습니다.
|
||||||
|
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_loop_storage_golden\\ -p:IntermediateOutputPath=obj\\verify_loop_storage_golden\\` 경고 0 / 오류 0
|
||||||
|
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentLoopDiagnosticsFormatterTests|ChatStorageServiceTests|HtmlSkillGoldenReportTests|DocxSkillGoldenDocumentTests|AgentMessageInvariantHelperTests|PptxSkillGoldenDeckTests|ExcelSkillGoldenWorkbookTests" -p:OutputPath=bin\\verify_loop_storage_golden_tests\\ -p:IntermediateOutputPath=obj\\verify_loop_storage_golden_tests\\` 통과 10
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
using AxCopilot.Services.Agent;
|
||||||
|
using FluentAssertions;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace AxCopilot.Tests.Services;
|
||||||
|
|
||||||
|
public class AgentLoopDiagnosticsFormatterTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void BuildCompactionCompleteMessage_ShouldIncludeStageSummaryAndSavedTokens()
|
||||||
|
{
|
||||||
|
var result = new ContextCompactionResult
|
||||||
|
{
|
||||||
|
BeforeTokens = 18_400,
|
||||||
|
AfterTokens = 7_900,
|
||||||
|
};
|
||||||
|
result.AppliedStages.Add("tool-result");
|
||||||
|
result.AppliedStages.Add("session-memory");
|
||||||
|
|
||||||
|
var message = AgentLoopDiagnosticsFormatter.BuildCompactionCompleteMessage(result);
|
||||||
|
|
||||||
|
message.Should().Be("컨텍스트 압축 완료 — tool-result -> session-memory · 10.3K tokens 절감");
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildQueryViewSummary_ShouldIncludeWindowAndPreviewReuseMetrics()
|
||||||
|
{
|
||||||
|
var result = new AgentQueryContextWindowResult
|
||||||
|
{
|
||||||
|
Messages = new(),
|
||||||
|
SourceMessageCount = 34,
|
||||||
|
ViewMessageCount = 12,
|
||||||
|
WindowStartIndex = 21,
|
||||||
|
BoundaryApplied = true,
|
||||||
|
ToolPairExpanded = true,
|
||||||
|
PreservedToolPairCount = 2,
|
||||||
|
TruncatedToolResultCount = 3,
|
||||||
|
ReusedToolResultPreviewCount = 4,
|
||||||
|
TokensBeforeBudget = 8_600,
|
||||||
|
TokensAfterBudget = 6_100,
|
||||||
|
};
|
||||||
|
|
||||||
|
var summary = AgentLoopDiagnosticsFormatter.BuildQueryViewSummary(result);
|
||||||
|
|
||||||
|
summary.Should().Be("query-view 34->12, start=21, pairs=2, tool_result_budget=3, tool_result_preview_reuse=4, tokens 8600->6100");
|
||||||
|
}
|
||||||
|
}
|
||||||
65
src/AxCopilot.Tests/Services/ChatStorageServiceTests.cs
Normal file
65
src/AxCopilot.Tests/Services/ChatStorageServiceTests.cs
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
using System.Text.Json;
|
||||||
|
using System.IO;
|
||||||
|
using AxCopilot.Models;
|
||||||
|
using AxCopilot.Services;
|
||||||
|
using FluentAssertions;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace AxCopilot.Tests.Services;
|
||||||
|
|
||||||
|
public class ChatStorageServiceTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public void Load_ShouldRestoreMissingToolResultPreviewFromLegacyStoredConversation()
|
||||||
|
{
|
||||||
|
var conversationId = "legacy-preview-" + Guid.NewGuid().ToString("N");
|
||||||
|
var storagePath = Path.Combine(
|
||||||
|
Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData),
|
||||||
|
"AxCopilot",
|
||||||
|
"conversations",
|
||||||
|
conversationId + ".axchat");
|
||||||
|
Directory.CreateDirectory(Path.GetDirectoryName(storagePath)!);
|
||||||
|
|
||||||
|
var conversation = new ChatConversation
|
||||||
|
{
|
||||||
|
Id = conversationId,
|
||||||
|
Title = "Legacy conversation",
|
||||||
|
Messages = new()
|
||||||
|
{
|
||||||
|
new ChatMessage
|
||||||
|
{
|
||||||
|
MsgId = "tool-result-1",
|
||||||
|
Role = "user",
|
||||||
|
Content = $$"""{"type":"tool_result","tool_use_id":"call-legacy","tool_name":"file_read","content":"{{new string('L', 900)}}" }""",
|
||||||
|
QueryPreviewContent = null,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
var json = JsonSerializer.Serialize(conversation);
|
||||||
|
CryptoService.EncryptToFile(storagePath, json);
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var storage = new ChatStorageService();
|
||||||
|
|
||||||
|
var loaded = storage.Load(conversationId);
|
||||||
|
|
||||||
|
loaded.Should().NotBeNull();
|
||||||
|
loaded!.Messages.Should().ContainSingle();
|
||||||
|
loaded.Messages[0].QueryPreviewContent.Should().NotBeNullOrWhiteSpace();
|
||||||
|
loaded.Messages[0].QueryPreviewContent.Should().Contain("call-legacy");
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var storage = new ChatStorageService();
|
||||||
|
storage.Delete(conversationId);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
149
src/AxCopilot.Tests/Services/DocxSkillGoldenDocumentTests.cs
Normal file
149
src/AxCopilot.Tests/Services/DocxSkillGoldenDocumentTests.cs
Normal file
@@ -0,0 +1,149 @@
|
|||||||
|
using System.IO;
|
||||||
|
using System.Text.Json;
|
||||||
|
using AxCopilot.Services.Agent;
|
||||||
|
using DocumentFormat.OpenXml.Packaging;
|
||||||
|
using DocumentFormat.OpenXml.Wordprocessing;
|
||||||
|
using FluentAssertions;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace AxCopilot.Tests.Services;
|
||||||
|
|
||||||
|
public class DocxSkillGoldenDocumentTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public async Task ExecuteAsync_WithStrongBusinessDocument_ShouldReturnStableQualitySummary()
|
||||||
|
{
|
||||||
|
var workDir = Path.Combine(Path.GetTempPath(), "ax-docx-golden-" + Guid.NewGuid().ToString("N"));
|
||||||
|
Directory.CreateDirectory(workDir);
|
||||||
|
var templatePath = Path.Combine(workDir, "golden-template.docx");
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
using (var template = WordprocessingDocument.Create(templatePath, DocumentFormat.OpenXml.WordprocessingDocumentType.Document))
|
||||||
|
{
|
||||||
|
var mainPart = template.AddMainDocumentPart();
|
||||||
|
mainPart.Document = new Document(new Body(
|
||||||
|
new Paragraph(
|
||||||
|
new ParagraphProperties(new ParagraphStyleId { Val = "Title" }),
|
||||||
|
new Run(new Text("Template Title"))),
|
||||||
|
new SectionProperties(
|
||||||
|
new PageSize { Width = 15840U, Height = 12240U },
|
||||||
|
new PageMargin { Top = 1080, Right = 1080, Bottom = 1080, Left = 1080 })));
|
||||||
|
var stylesPart = mainPart.AddNewPart<StyleDefinitionsPart>();
|
||||||
|
stylesPart.Styles = new Styles(
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "Title" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "Title", CustomStyle = true },
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "Subtitle" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "Subtitle", CustomStyle = true },
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "Heading1" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "Heading1", CustomStyle = true },
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "Heading2" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "Heading2", CustomStyle = true },
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "Normal" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "Normal", CustomStyle = true },
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "Quote" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "Quote", CustomStyle = true },
|
||||||
|
new Style(
|
||||||
|
new StyleName { Val = "TableHeader" })
|
||||||
|
{ Type = StyleValues.Paragraph, StyleId = "TableHeader", CustomStyle = true });
|
||||||
|
stylesPart.Styles.Save();
|
||||||
|
mainPart.Document.Save();
|
||||||
|
}
|
||||||
|
|
||||||
|
var tool = new DocumentAssemblerTool();
|
||||||
|
var context = new AgentContext
|
||||||
|
{
|
||||||
|
WorkFolder = workDir,
|
||||||
|
Permission = "Auto",
|
||||||
|
OperationMode = "external",
|
||||||
|
};
|
||||||
|
|
||||||
|
var args = JsonDocument.Parse(
|
||||||
|
"""
|
||||||
|
{
|
||||||
|
"path": "board-golden.docx",
|
||||||
|
"title": "Board Decision Pack",
|
||||||
|
"format": "docx",
|
||||||
|
"toc": true,
|
||||||
|
"cover_subtitle": "Q2 Operating Review",
|
||||||
|
"template_path": "golden-template.docx",
|
||||||
|
"header": "AX Copilot Board Pack",
|
||||||
|
"footer": "Internal Use Only {page}",
|
||||||
|
"page_numbers": true,
|
||||||
|
"style_map": {
|
||||||
|
"title": "Title",
|
||||||
|
"heading1": "Heading1",
|
||||||
|
"heading2": "Heading2",
|
||||||
|
"body": "Normal",
|
||||||
|
"cover_subtitle": "Subtitle",
|
||||||
|
"callout": "Quote",
|
||||||
|
"table_header": "TableHeader"
|
||||||
|
},
|
||||||
|
"sections": [
|
||||||
|
{
|
||||||
|
"heading": "Executive Summary",
|
||||||
|
"level": 1,
|
||||||
|
"content": "<p>The executive summary confirms that the pilot wave improved margin, service stability, and delivery control without creating operational drag. The decision can now move from exploration to approval because the evidence base is consistent across finance, operations, and PMO reporting.</p><p>Management recommends approving the next rollout wave now, while preserving two governance checkpoints. This maintains momentum and protects execution quality at the same time.</p>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"heading": "Current State",
|
||||||
|
"level": 1,
|
||||||
|
"content": "<p>The current state review shows that the major execution bottlenecks remain concentrated in a small number of handoffs and exception paths. Standard routing and weekly operating reviews have already reduced escalation rates and rework demand in the pilot teams.</p><p>Commercial performance also remains healthy. Revenue is above plan, margin is improving, and customer stability indicators remain within target even while the operating model is changing.</p><ul><li>Lead time improved across every pilot team.</li><li>Margin uplift remained visible for the full pilot period.</li><li>Escalation volume declined after governance tightened.</li></ul>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"heading": "Evidence Base",
|
||||||
|
"level": 1,
|
||||||
|
"content": "<p>The evidence base combines PMO trackers, close-cycle finance observations, and field feedback from the operating teams. Each source points to the same conclusion: the new operating model is easier to govern and more resilient under volume pressure.</p><div class=\"callout-info\"><strong>Key evidence</strong><p>The most important pattern is consistency. Margin, lead time, and service stability all move in the right direction, and none of the pilot teams show a material negative trade-off.</p></div><table><tr><th>Source</th><th>Observation</th></tr><tr><td>Finance close</td><td>Margin improved by 4.2 points in the pilot teams.</td></tr><tr><td>PMO tracker</td><td>Lead time dropped from 12 days to 9.8 days.</td></tr></table>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"heading": "Options",
|
||||||
|
"level": 1,
|
||||||
|
"content": "<p>Management assessed three practical rollout options. A pilot-only path is the safest near-term choice but delays value capture. A phased rollout balances control and speed. A full rollout is the fastest path but also creates the largest adoption risk.</p><p>The phased route is therefore the recommended choice because it captures the majority of the value while keeping governance strong and transparent.</p>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"heading": "Recommendation",
|
||||||
|
"level": 1,
|
||||||
|
"content": "<p>The board should approve the next rollout wave and release the phase-2 budget immediately. The pilot evidence is already sufficient, and the proposed governance cadence makes the residual risks manageable.</p><p>Immediate approval avoids losing execution momentum and allows the PMO to lock resources, communication, and accountability while the organization is still aligned around the pilot results.</p>"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"heading": "Appendix",
|
||||||
|
"level": 1,
|
||||||
|
"content": "<p>The appendix captures supporting notes, reference observations, and detailed KPI descriptions. It is included to preserve traceability from the board recommendation back to the underlying operational evidence.</p><p>This supporting material is intentionally concise because the core argument already appears in the main body of the document. The appendix exists for auditability and follow-up, not as a substitute for the main recommendation.</p>"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
""").RootElement;
|
||||||
|
|
||||||
|
var result = await tool.ExecuteAsync(args, context, CancellationToken.None);
|
||||||
|
|
||||||
|
result.Success.Should().BeTrue();
|
||||||
|
result.Output.Should().Contain("DOCX 품질 리뷰:");
|
||||||
|
result.Output.Should().Contain("이슈 0");
|
||||||
|
result.Output.Should().Contain("Repair guide: none");
|
||||||
|
|
||||||
|
var outputPath = Path.Combine(workDir, "board-golden.docx");
|
||||||
|
File.Exists(outputPath).Should().BeTrue();
|
||||||
|
|
||||||
|
using var doc = WordprocessingDocument.Open(outputPath, false);
|
||||||
|
doc.MainDocumentPart!.Document.Body!.InnerText.Should().Contain("Q2 Operating Review");
|
||||||
|
doc.MainDocumentPart.Document.Body.Descendants<FieldCode>().Should().Contain(field => field.Text.Contains("TOC"));
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (Directory.Exists(workDir))
|
||||||
|
Directory.Delete(workDir, true);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
145
src/AxCopilot.Tests/Services/HtmlSkillGoldenReportTests.cs
Normal file
145
src/AxCopilot.Tests/Services/HtmlSkillGoldenReportTests.cs
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
using System.IO;
|
||||||
|
using System.Text.Json;
|
||||||
|
using AxCopilot.Services.Agent;
|
||||||
|
using FluentAssertions;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace AxCopilot.Tests.Services;
|
||||||
|
|
||||||
|
public class HtmlSkillGoldenReportTests
|
||||||
|
{
|
||||||
|
[Fact]
|
||||||
|
public async Task ExecuteAsync_WithStrongBoardReport_ShouldReturnStableQualitySummary()
|
||||||
|
{
|
||||||
|
var workDir = Path.Combine(Path.GetTempPath(), "ax-html-golden-" + Guid.NewGuid().ToString("N"));
|
||||||
|
Directory.CreateDirectory(workDir);
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var tool = new HtmlSkill();
|
||||||
|
var context = new AgentContext
|
||||||
|
{
|
||||||
|
WorkFolder = workDir,
|
||||||
|
Permission = "Auto",
|
||||||
|
OperationMode = "external",
|
||||||
|
};
|
||||||
|
|
||||||
|
var args = JsonDocument.Parse(
|
||||||
|
"""
|
||||||
|
{
|
||||||
|
"path": "board-golden.html",
|
||||||
|
"title": "Board Operating Review",
|
||||||
|
"body": "",
|
||||||
|
"toc": true,
|
||||||
|
"print": true,
|
||||||
|
"print_header": "AX Copilot Board Pack",
|
||||||
|
"print_footer": "Internal Use Only",
|
||||||
|
"cover": {
|
||||||
|
"title": "Board Operating Review",
|
||||||
|
"subtitle": "Q2 decision and execution pack",
|
||||||
|
"author": "AX Copilot",
|
||||||
|
"date": "2026-04-15"
|
||||||
|
},
|
||||||
|
"sections": [
|
||||||
|
{ "type": "heading", "level": 2, "text": "Executive Summary" },
|
||||||
|
{ "type": "paragraph", "text": "The operating review shows that the pilot wave is delivering the expected revenue resilience, service stability, and margin protection. The board can now decide on the next rollout wave with evidence that the core execution risks are already bounded." },
|
||||||
|
{ "type": "paragraph", "text": "Management recommends approving the second rollout wave now, while preserving two governance checkpoints. This keeps momentum high without overloading delivery teams or introducing avoidable execution variance." },
|
||||||
|
{
|
||||||
|
"type": "board_report",
|
||||||
|
"title": "Board Ask",
|
||||||
|
"decision": "Approve the next rollout wave",
|
||||||
|
"recommendation": "Release the phase-2 budget and retain the PMO checkpoint cadence",
|
||||||
|
"rationale": "Pilot teams met the target on margin, lead time, and customer stability.",
|
||||||
|
"metrics": [
|
||||||
|
{ "label": "Margin", "value": "+4.2pt", "note": "pilot mix" },
|
||||||
|
{ "label": "Lead Time", "value": "-18%", "note": "handoff removal" }
|
||||||
|
],
|
||||||
|
"risks": ["Regional adoption lag", "Temporary staffing bottleneck"],
|
||||||
|
"next_steps": ["Approve budget", "Launch the next wave", "Review outcomes at week 8"]
|
||||||
|
},
|
||||||
|
{ "type": "heading", "level": 2, "text": "Current State" },
|
||||||
|
{ "type": "paragraph", "text": "The current-state assessment confirms that process variation and rework are concentrated in a small number of high-volume handoffs. The pilot proved that standard routing and weekly governance significantly reduce friction without reducing delivery throughput." },
|
||||||
|
{ "type": "paragraph", "text": "Commercial performance also remains stable. Revenue is above plan, service quality indicators remain healthy, and the cost profile is improving because exceptions are being handled earlier in the workflow." },
|
||||||
|
{
|
||||||
|
"type": "comparison",
|
||||||
|
"title": "Options",
|
||||||
|
"items": [
|
||||||
|
{ "name": "Pilot only", "summary": "Protects near-term stability", "pros": "Lowest change risk", "cons": "Defers value capture", "verdict": "Too cautious" },
|
||||||
|
{ "name": "Phased rollout", "summary": "Balances speed and control", "pros": "Captures value with governance", "cons": "Needs PMO discipline", "verdict": "Recommended" },
|
||||||
|
{ "name": "Full rollout", "summary": "Fastest expansion path", "pros": "Maximum pace", "cons": "Highest adoption risk", "verdict": "Too disruptive" }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{ "type": "heading", "level": 2, "text": "Evidence Base" },
|
||||||
|
{ "type": "paragraph", "text": "The evidence base combines performance data, PMO logs, and finance-close observations. The most important pattern is consistency: each pilot location shows the same direction of improvement on cycle time, margin, and escalation frequency." },
|
||||||
|
{ "type": "paragraph", "text": "This makes the decision less about whether the model works and more about how quickly to scale it. The remaining debate is sequencing, not strategic validity." },
|
||||||
|
{
|
||||||
|
"type": "evidence_cards",
|
||||||
|
"title": "Evidence",
|
||||||
|
"items": [
|
||||||
|
{ "title": "Margin uplift", "detail": "+4.2 point improvement after standard routing", "source": "Finance close", "tag": "KPI" },
|
||||||
|
{ "title": "Lead time", "detail": "Average lead time reduced from 12 days to 9.8 days", "source": "PMO tracker", "tag": "Ops" }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{ "type": "heading", "level": 2, "text": "Roadmap" },
|
||||||
|
{ "type": "paragraph", "text": "Execution should proceed in three steps. First, management locks the scope and budget. Second, the PMO extends the governance cadence to the next wave. Third, executive checkpoints validate performance before the final scale decision." },
|
||||||
|
{ "type": "paragraph", "text": "This sequence keeps accountability visible to the board and prevents the organization from treating rollout as a one-time launch. The process is intentionally designed as a controlled scaling motion." },
|
||||||
|
{
|
||||||
|
"type": "roadmap",
|
||||||
|
"title": "Execution Roadmap",
|
||||||
|
"phases": [
|
||||||
|
{ "title": "Approve", "detail": "Lock funding and governance scope", "timeline": "Week 1", "owner": "Board + COO" },
|
||||||
|
{ "title": "Launch", "detail": "Activate the next regional wave", "timeline": "Weeks 2-6", "owner": "PMO" },
|
||||||
|
{ "title": "Review", "detail": "Confirm margin and service outcomes", "timeline": "Week 8", "owner": "Executive Committee" }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{ "type": "heading", "level": 2, "text": "Recommendation" },
|
||||||
|
{ "type": "paragraph", "text": "The recommended decision is to approve the next rollout wave now. The organization has enough evidence to move forward, and the proposed governance checkpoints are sufficient to manage the residual risks." },
|
||||||
|
{ "type": "paragraph", "text": "Delaying the decision would reduce momentum without creating meaningfully better information. A structured approval now is the most value-protective option." },
|
||||||
|
{
|
||||||
|
"type": "decision_summary",
|
||||||
|
"title": "Decision Summary",
|
||||||
|
"decision": "Approve the next rollout wave and release the phase-2 budget.",
|
||||||
|
"rationale": "Pilot evidence shows stable value capture and manageable execution risk.",
|
||||||
|
"actions": ["Approve budget release", "Launch phase-2 governance cadence", "Review outcomes at week 8"]
|
||||||
|
},
|
||||||
|
{ "type": "heading", "level": 2, "text": "Appendix" },
|
||||||
|
{ "type": "paragraph", "text": "The appendix contains supporting observations, KPI references, and delivery assumptions. It is included to give the board direct traceability from the recommendation to the underlying evidence." },
|
||||||
|
{ "type": "paragraph", "text": "Supporting material is intentionally concise. The core case for action already appears in the main body, and the appendix exists to preserve auditability rather than to carry the main argument." },
|
||||||
|
{
|
||||||
|
"type": "table",
|
||||||
|
"headers": ["Evidence", "Observation"],
|
||||||
|
"rows": [
|
||||||
|
["Finance close", "Margin expansion sustained for the full pilot period"],
|
||||||
|
["PMO tracker", "Escalation frequency declined across every pilot team"]
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
""").RootElement;
|
||||||
|
|
||||||
|
var result = await tool.ExecuteAsync(args, context, CancellationToken.None);
|
||||||
|
|
||||||
|
result.Success.Should().BeTrue();
|
||||||
|
result.Output.Should().Contain("Quality score");
|
||||||
|
result.Output.Should().Contain("Needs work: none");
|
||||||
|
result.Output.Should().Contain("Repair guide: none");
|
||||||
|
|
||||||
|
var html = File.ReadAllText(Path.Combine(workDir, "board-golden.html"));
|
||||||
|
html.Should().Contain("decision-summary");
|
||||||
|
html.Should().Contain("evidence-cards");
|
||||||
|
html.Should().Contain("board-report-panel");
|
||||||
|
html.Should().Contain("roadmap-block");
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
if (Directory.Exists(workDir))
|
||||||
|
Directory.Delete(workDir, true);
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
namespace AxCopilot.Services.Agent;
|
||||||
|
|
||||||
|
internal static class AgentLoopDiagnosticsFormatter
|
||||||
|
{
|
||||||
|
public static string BuildCompactionCompleteMessage(ContextCompactionResult result)
|
||||||
|
{
|
||||||
|
var compactSummary = !string.IsNullOrWhiteSpace(result.StageSummary)
|
||||||
|
? result.StageSummary
|
||||||
|
: "기본";
|
||||||
|
return $"컨텍스트 압축 완료 — {compactSummary} · {Services.TokenEstimator.Format(result.SavedTokens)} tokens 절감";
|
||||||
|
}
|
||||||
|
|
||||||
|
public static string BuildQueryViewSummary(AgentQueryContextWindowResult queryView)
|
||||||
|
{
|
||||||
|
return
|
||||||
|
$"query-view {queryView.SourceMessageCount}->{queryView.ViewMessageCount}, " +
|
||||||
|
$"start={queryView.WindowStartIndex}, " +
|
||||||
|
$"pairs={queryView.PreservedToolPairCount}, " +
|
||||||
|
$"tool_result_budget={queryView.TruncatedToolResultCount}, " +
|
||||||
|
$"tool_result_preview_reuse={queryView.ReusedToolResultPreviewCount}, " +
|
||||||
|
$"tokens {queryView.TokensBeforeBudget}->{queryView.TokensAfterBudget}";
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -475,13 +475,10 @@ public partial class AgentLoopService
|
|||||||
if (compactionResult.Changed)
|
if (compactionResult.Changed)
|
||||||
{
|
{
|
||||||
MarkRunPostCompaction(compactionResult, runState);
|
MarkRunPostCompaction(compactionResult, runState);
|
||||||
var compactSummary = !string.IsNullOrWhiteSpace(compactionResult.StageSummary)
|
|
||||||
? compactionResult.StageSummary
|
|
||||||
: "기본";
|
|
||||||
EmitEvent(
|
EmitEvent(
|
||||||
AgentEventType.Thinking,
|
AgentEventType.Thinking,
|
||||||
"",
|
"",
|
||||||
$"컨텍스트 압축 완료 — {compactSummary} · {Services.TokenEstimator.Format(compactionResult.SavedTokens)} tokens 절감");
|
AgentLoopDiagnosticsFormatter.BuildCompactionCompleteMessage(compactionResult));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -505,19 +502,12 @@ public partial class AgentLoopService
|
|||||||
var queryMessages = queryView.Messages;
|
var queryMessages = queryView.Messages;
|
||||||
if (queryView.BoundaryApplied || queryView.ToolPairExpanded || queryView.TruncatedToolResultCount > 0)
|
if (queryView.BoundaryApplied || queryView.ToolPairExpanded || queryView.TruncatedToolResultCount > 0)
|
||||||
{
|
{
|
||||||
var queryViewSummary =
|
|
||||||
$"query-view {queryView.SourceMessageCount}->{queryView.ViewMessageCount}, " +
|
|
||||||
$"start={queryView.WindowStartIndex}, " +
|
|
||||||
$"pairs={queryView.PreservedToolPairCount}, " +
|
|
||||||
$"tool_result_budget={queryView.TruncatedToolResultCount}, " +
|
|
||||||
$"tool_result_preview_reuse={queryView.ReusedToolResultPreviewCount}, " +
|
|
||||||
$"tokens {queryView.TokensBeforeBudget}->{queryView.TokensAfterBudget}";
|
|
||||||
WorkflowLogService.LogTransition(
|
WorkflowLogService.LogTransition(
|
||||||
_conversationId,
|
_conversationId,
|
||||||
_currentRunId,
|
_currentRunId,
|
||||||
iteration,
|
iteration,
|
||||||
"query_view",
|
"query_view",
|
||||||
queryViewSummary);
|
AgentLoopDiagnosticsFormatter.BuildQueryViewSummary(queryView));
|
||||||
}
|
}
|
||||||
|
|
||||||
var isDebugLog = string.Equals(_settings.Settings.Llm.AgentLogLevel, "debug", StringComparison.OrdinalIgnoreCase);
|
var isDebugLog = string.Equals(_settings.Settings.Llm.AgentLogLevel, "debug", StringComparison.OrdinalIgnoreCase);
|
||||||
|
|||||||
@@ -78,7 +78,10 @@ public class ChatStorageService : IChatStorageService
|
|||||||
{
|
{
|
||||||
if (!File.Exists(path)) return null;
|
if (!File.Exists(path)) return null;
|
||||||
var json = CryptoService.DecryptFromFile(path);
|
var json = CryptoService.DecryptFromFile(path);
|
||||||
return JsonSerializer.Deserialize<ChatConversation>(json, JsonOpts);
|
var conversation = JsonSerializer.Deserialize<ChatConversation>(json, JsonOpts);
|
||||||
|
if (conversation != null)
|
||||||
|
AxCopilot.Services.Agent.AgentMessageInvariantHelper.PopulateMissingToolResultPreviews(conversation.Messages);
|
||||||
|
return conversation;
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
|
|||||||
Reference in New Issue
Block a user