?? ?? tool_result preview ??? ???? golden workbook ??? ??

?? ??
- ??, ??, ?? ?? tool_result preview? ??? ??? ???? replacement state? ?? ??? ? ??? ??? ?? ??? ??
- XLSX ?? ?? ??? ?? ?? ?? workbook golden ??? ??? summary/dashboard/detail ??? ????? ??

?? ????
- AgentMessageInvariantHelper? synthetic tool_result preview ?? ??? ??? QueryPreviewContent? ?? ?? ?? ??? tool_use_id, tool_name, ??? content/output/error ?? preview? ????? ??
- BuildToolResultPreviewMap? ?? preview? ?? ???? ?? ?? synthetic preview? ?? ??? ??
- AgentMessageInvariantHelperTests? ??? preview? ?? long tool_result? synthetic preview? ???? ??? explicit preview ?? ?? ??? ?? ??
- AgentQueryContextBuilderTests? synthetic preview? query view ?? ? ?? ???? ???? ????? ??
- ExcelSkillGoldenWorkbookTests? ??? summary/dashboard/detail, formula, data validation, conditional formatting? ??? ?? ?? workbook? Needs work: none ? Repair guide: none? ????? ??
- README.md? docs/DEVELOPMENT.md? 2026-04-15 09:36 (KST) ?? ?? ??? ?? ??? ??

?? ??
- dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\ : ?? 0 / ?? 0
- dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\ : ?? 10
This commit is contained in:
2026-04-15 09:11:56 +09:00
parent ff29a83039
commit 8530ec956a
6 changed files with 299 additions and 1 deletions

View File

@@ -1925,3 +1925,10 @@ MIT License
- 테스트로 [AgentQueuedCommandProjectorTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentQueuedCommandProjectorTests.cs), [DeckQualityReviewServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/DeckQualityReviewServiceTests.cs), [ArtifactQualityReviewServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ArtifactQualityReviewServiceTests.cs), [ArtifactRepairGuideServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ArtifactRepairGuideServiceTests.cs), [DeckRepairGuideServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/DeckRepairGuideServiceTests.cs)를 확장했습니다.
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_loop_doc_finish2\\ -p:IntermediateOutputPath=obj\\verify_loop_doc_finish2\\` 경고 0 / 오류 0
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentQueuedCommandProjectorTests|AgentCommandQueueTests|ArtifactQualityReviewServiceTests|ArtifactRepairGuideServiceTests|DeckQualityReviewServiceTests|DeckRepairGuideServiceTests|PptxSkillGoldenDeckTests|ExcelSkillDashboardSummaryTests" -p:OutputPath=bin\\verify_loop_doc_finish2_tests\\ -p:IntermediateOutputPath=obj\\verify_loop_doc_finish2_tests\\` 통과 25
업데이트: 2026-04-15 09:36 (KST)
- 장기 세션에서 `tool_result` preview가 통째로 사라진 경우를 대비해 [AgentMessageInvariantHelper.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentMessageInvariantHelper.cs)에 synthetic preview 복원 경로를 추가했습니다. 이제 저장/재개/분기 이후 기존 preview가 하나도 남지 않아도 `tool_use_id`, `tool_name`, 축약된 `content`를 기반으로 query preview를 다시 만들 수 있습니다.
- [AgentQueryContextBuilder.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentQueryContextBuilder.cs)가 이 복원 경로를 그대로 활용하도록 회귀를 보강했고, 새 테스트 [AgentMessageInvariantHelperTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentMessageInvariantHelperTests.cs), [AgentQueryContextBuilderTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentQueryContextBuilderTests.cs)로 synthetic preview 생성과 query view 반영을 고정했습니다.
- 문서 golden 회귀도 확대했습니다. [ExcelSkillGoldenWorkbookTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ExcelSkillGoldenWorkbookTests.cs)는 summary/dashboard/detail 구조와 formula, data validation, conditional formatting이 모두 들어간 운영 리뷰 workbook이 `Needs work: none``Repair guide: none`을 유지하는지 검증합니다.
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\` 경고 0 / 오류 0
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\` 통과 10

View File

@@ -996,3 +996,10 @@ UI ?붿옄???€洹쒕え 由ы뙥?좊쭅 ???꾪뿕 ?묒뾽 ??湲곕줉???덉쟾
- 테스트는 [AgentQueuedCommandProjectorTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentQueuedCommandProjectorTests.cs), [DeckQualityReviewServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/DeckQualityReviewServiceTests.cs), [ArtifactQualityReviewServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ArtifactQualityReviewServiceTests.cs), [ArtifactRepairGuideServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ArtifactRepairGuideServiceTests.cs), [DeckRepairGuideServiceTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/DeckRepairGuideServiceTests.cs)를 확장해 회귀를 고정했습니다.
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_loop_doc_finish2\\ -p:IntermediateOutputPath=obj\\verify_loop_doc_finish2\\` 경고 0 / 오류 0
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentQueuedCommandProjectorTests|AgentCommandQueueTests|ArtifactQualityReviewServiceTests|ArtifactRepairGuideServiceTests|DeckQualityReviewServiceTests|DeckRepairGuideServiceTests|PptxSkillGoldenDeckTests|ExcelSkillDashboardSummaryTests" -p:OutputPath=bin\\verify_loop_doc_finish2_tests\\ -p:IntermediateOutputPath=obj\\verify_loop_doc_finish2_tests\\` 통과 25
업데이트: 2026-04-15 09:36 (KST)
- `tool_result` replacement state의 마지막 빈틈을 메웠습니다. [AgentMessageInvariantHelper.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot/Services/Agent/AgentMessageInvariantHelper.cs)는 기존 `QueryPreviewContent`가 하나도 없는 경우에도 `tool_use_id`, `tool_name`, 축약된 `content/output/error`를 기반으로 synthetic preview를 생성합니다. 이로써 저장/재개/분기 이후 preview가 완전히 유실된 세션에서도 다시 query preview를 만들 수 있습니다.
- [AgentQueryContextBuilderTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentQueryContextBuilderTests.cs)와 새 [AgentMessageInvariantHelperTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/AgentMessageInvariantHelperTests.cs)는 preview가 없는 tool_result가 synthetic preview로 복원되고, query view 생성 시에도 같은 preview가 실제 반영되는지 회귀 검증합니다.
- 문서 golden 회귀도 한 단계 더 올렸습니다. 새 [ExcelSkillGoldenWorkbookTests.cs](/E:/AX%20Copilot%20-%20Codex/src/AxCopilot.Tests/Services/ExcelSkillGoldenWorkbookTests.cs)는 summary/dashboard/detail 구조와 formula, data validation, conditional formatting이 모두 포함된 운영 리뷰 workbook이 `Needs work: none`, `Repair guide: none`을 유지하는지 확인합니다.
- 검증: `dotnet build src/AxCopilot/AxCopilot.csproj -c Release -v minimal -p:OutputPath=bin\\verify_preview_golden_finish\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish\\` 경고 0 / 오류 0
- 검증: `dotnet test src/AxCopilot.Tests/AxCopilot.Tests.csproj -c Release -v minimal --filter "AgentMessageInvariantHelperTests|AgentQueryContextBuilderTests|AgentQueuedCommandProjectorTests|ExcelSkillGoldenWorkbookTests|ExcelSkillDashboardSummaryTests|PptxSkillGoldenDeckTests" -p:OutputPath=bin\\verify_preview_golden_finish_tests\\ -p:IntermediateOutputPath=obj\\verify_preview_golden_finish_tests\\` 통과 10

View File

@@ -0,0 +1,58 @@
using AxCopilot.Models;
using AxCopilot.Services.Agent;
using FluentAssertions;
using Xunit;
namespace AxCopilot.Tests.Services;
public class AgentMessageInvariantHelperTests
{
[Fact]
public void PopulateMissingToolResultPreviews_ShouldSynthesizePreview_WhenNoStoredPreviewExists()
{
var longContent = string.Join(' ', Enumerable.Repeat("Detailed output row for recovery.", 40));
var messages = new List<ChatMessage>
{
new()
{
MsgId = "tool-result-1",
Role = "user",
Content = $$"""{"type":"tool_result","tool_use_id":"call-synth","tool_name":"file_read","content":"{{longContent}}"}"""
}
};
var changed = AgentMessageInvariantHelper.PopulateMissingToolResultPreviews(messages);
changed.Should().BeTrue();
messages[0].QueryPreviewContent.Should().NotBeNullOrWhiteSpace();
messages[0].QueryPreviewContent.Should().Contain("call-synth");
messages[0].QueryPreviewContent.Should().Contain("...");
messages[0].QueryPreviewContent.Should().NotContain(longContent);
}
[Fact]
public void BuildToolResultPreviewMap_ShouldPreferExplicitPreview_WhenAvailable()
{
var explicitPreview = """{"type":"tool_result","tool_use_id":"call-explicit","tool_name":"file_read","content":"preview"}""";
var messages = new List<ChatMessage>
{
new()
{
MsgId = "tool-result-1",
Role = "user",
Content = """{"type":"tool_result","tool_use_id":"call-explicit","tool_name":"file_read","content":"long output content"}""",
QueryPreviewContent = explicitPreview
},
new()
{
MsgId = "tool-result-2",
Role = "user",
Content = """{"type":"tool_result","tool_use_id":"call-explicit","tool_name":"file_read","content":"another long output content"}"""
}
};
var map = AgentMessageInvariantHelper.BuildToolResultPreviewMap(messages);
map["call-explicit"].Should().Be(explicitPreview);
}
}

View File

@@ -39,4 +39,33 @@ public class AgentQueryContextBuilderTests
sourceMessages[1].QueryPreviewContent.Should().Be(sourceMessages[0].QueryPreviewContent);
result.Messages[1].QueryPreviewContent.Should().Be(sourceMessages[0].QueryPreviewContent);
}
[Fact]
public void Build_ShouldSynthesizeToolResultPreview_WhenNoStoredPreviewExists()
{
var longContent = string.Join(' ', Enumerable.Repeat("long tool output", 120));
var sourceMessages = new List<ChatMessage>
{
new()
{
MsgId = "tool-source-1",
Role = "user",
Content = $$"""{"type":"tool_result","tool_use_id":"call-synth-view","tool_name":"file_read","content":"{{longContent}}"}"""
},
new()
{
MsgId = "tail-1",
Role = "assistant",
Content = "recent tail"
}
};
var result = AgentQueryContextBuilder.Build(sourceMessages);
sourceMessages[0].QueryPreviewContent.Should().NotBeNullOrWhiteSpace();
sourceMessages[0].QueryPreviewContent.Should().Contain("call-synth-view");
result.Messages.Should().Contain(message =>
message.QueryPreviewContent != null &&
message.QueryPreviewContent.Contains("call-synth-view", StringComparison.OrdinalIgnoreCase));
}
}

View File

@@ -0,0 +1,94 @@
using System.IO;
using System.Text.Json;
using AxCopilot.Services.Agent;
using FluentAssertions;
using Xunit;
namespace AxCopilot.Tests.Services;
public class ExcelSkillGoldenWorkbookTests
{
[Fact]
public async Task ExecuteAsync_WithRichOperatingWorkbook_ShouldReturnStableQualitySummary()
{
var workDir = Path.Combine(Path.GetTempPath(), "ax-xlsx-golden-" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(workDir);
try
{
var tool = new ExcelSkill();
var context = new AgentContext
{
WorkFolder = workDir,
Permission = "Auto",
OperationMode = "external",
};
var args = JsonDocument.Parse(
"""
{
"path": "operating-golden.xlsx",
"summary_sheet": {
"name": "Summary",
"dashboard_sheet_name": "Dashboard",
"title": "Operating Review",
"decision_summary": [
{ "label": "Decision Ask", "value": "Approve phase-2 staffing", "owner": "COO" }
],
"scorecards": [
{ "label": "Revenue", "value": "128", "status": "On Track", "note": "Above plan" }
],
"dashboard_tiles": [
{ "label": "Funding Status", "value": "Ready", "status": "Green", "note": "Decision aligned" }
],
"trend_series": [
{ "label": "Revenue", "current": "128", "target": "125", "delta": "+3", "status": "Green" }
],
"variance_series": [
{ "label": "Opex", "actual": "84", "target": "82", "variance": "+2", "status": "Watch" }
],
"sheet_summaries": [
{ "sheet": "Revenue", "status": "Green", "summary": "Bookings remain above plan", "owner": "Sales Ops" },
{ "sheet": "Cost", "status": "Watch", "summary": "Contractor spend needs control", "owner": "Finance" }
],
"highlights": ["Revenue remains above plan"],
"actions": ["Approve phase-2 staffing"]
},
"sheets": [
{
"name": "Revenue",
"headers": ["Metric", "Value"],
"rows": [["Revenue", 128], ["Growth", "=B2/100"], ["Target Gap", "=B2-125"]],
"conditional_formats": [{ "range": "B2:B3", "type": "data_bar", "color": "2563EB" }]
},
{
"name": "Cost",
"headers": ["Metric", "Value"],
"rows": [["Opex", 84], ["Target", 82], ["Variance", "=B2-B3"]],
"data_validations": [{ "range": "A2:A3", "type": "list", "formula1": "\"Opex,Target\"", "allow_blank": false }]
}
]
}
""").RootElement;
var result = await tool.ExecuteAsync(args, context, CancellationToken.None);
result.Success.Should().BeTrue();
result.Output.Should().Contain("Quality score");
result.Output.Should().Contain("Needs work: none");
result.Output.Should().Contain("Repair guide: none");
File.Exists(Path.Combine(workDir, "operating-golden.xlsx")).Should().BeTrue();
}
finally
{
try
{
if (Directory.Exists(workDir))
Directory.Delete(workDir, true);
}
catch
{
}
}
}
}

View File

@@ -8,13 +8,17 @@ namespace AxCopilot.Services.Agent;
/// </summary>
internal static class AgentMessageInvariantHelper
{
private const int SyntheticPreviewTextLimit = 280;
public static Dictionary<string, string> BuildToolResultPreviewMap(IEnumerable<ChatMessage>? messages)
{
var previews = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (messages == null)
return previews;
foreach (var message in messages)
var bufferedMessages = messages as IList<ChatMessage> ?? messages.ToList();
foreach (var message in bufferedMessages)
{
if (string.IsNullOrWhiteSpace(message.QueryPreviewContent))
continue;
@@ -24,6 +28,16 @@ internal static class AgentMessageInvariantHelper
previews[toolResultId] = message.QueryPreviewContent!;
}
foreach (var message in bufferedMessages)
{
if (!TryGetToolResultId(message, out var toolResultId) || previews.ContainsKey(toolResultId))
continue;
if (!TryBuildSyntheticToolResultPreview(message, out var preview))
continue;
previews[toolResultId] = preview;
}
return previews;
}
@@ -164,4 +178,93 @@ internal static class AgentMessageInvariantHelper
doc?.Dispose();
}
}
internal static bool TryBuildSyntheticToolResultPreview(ChatMessage message, out string preview)
{
preview = "";
if (!TryGetToolResultId(message, out var toolResultId))
return false;
var content = message.Content ?? "";
if (string.IsNullOrWhiteSpace(content))
return false;
try
{
using var doc = JsonDocument.Parse(content);
var root = doc.RootElement;
var toolName = root.TryGetProperty("tool_name", out var toolNameEl)
? toolNameEl.GetString()
: null;
var previewText = ExtractPreviewText(root);
if (string.IsNullOrWhiteSpace(previewText))
previewText = "tool result available";
var payload = new Dictionary<string, object?>
{
["type"] = "tool_result",
["tool_use_id"] = toolResultId,
["tool_name"] = toolName,
["content"] = previewText
};
preview = JsonSerializer.Serialize(payload);
return true;
}
catch
{
return false;
}
}
private static string ExtractPreviewText(JsonElement root)
{
if (root.TryGetProperty("content", out var contentEl))
{
var preview = NormalizePreviewText(contentEl.ValueKind switch
{
JsonValueKind.String => contentEl.GetString() ?? "",
JsonValueKind.Object or JsonValueKind.Array => contentEl.GetRawText(),
JsonValueKind.Number or JsonValueKind.True or JsonValueKind.False => contentEl.ToString(),
_ => ""
});
if (!string.IsNullOrWhiteSpace(preview))
return preview;
}
if (root.TryGetProperty("output", out var outputEl))
{
var preview = NormalizePreviewText(outputEl.ValueKind switch
{
JsonValueKind.String => outputEl.GetString() ?? "",
JsonValueKind.Object or JsonValueKind.Array => outputEl.GetRawText(),
JsonValueKind.Number or JsonValueKind.True or JsonValueKind.False => outputEl.ToString(),
_ => ""
});
if (!string.IsNullOrWhiteSpace(preview))
return preview;
}
if (root.TryGetProperty("error", out var errorEl) && errorEl.ValueKind == JsonValueKind.String)
return NormalizePreviewText(errorEl.GetString() ?? "");
return "";
}
private static string NormalizePreviewText(string text)
{
if (string.IsNullOrWhiteSpace(text))
return "";
var collapsed = string.Join(" ", text
.Replace('\r', ' ')
.Replace('\n', ' ')
.Replace('\t', ' ')
.Split(' ', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries));
if (collapsed.Length <= SyntheticPreviewTextLimit)
return collapsed;
return collapsed[..(SyntheticPreviewTextLimit - 3)].TrimEnd() + "...";
}
}