Can You Trust AI at Work?
McKenzie Yastremski
Introduction
Artificial intelligence is becoming increasingly common in the workplace. Employees may use AI to summarize documents, answer questions, analyze information, and help make decisions. However, AI can sometimes provide answers that sound confident even when it does not have enough information to answer accurately. This can be a problem when the information involves workplace policies and employee responsibilities.
For this assignment, I tested Western Carolina University's Amplify AI platform by asking five questions about WCU's remote-work policy and North Carolina's state telework policy. First, I asked the questions without providing the policy documents. Then, I started a separate conversation and provided policy-related source material before asking questions about remote work again.
The purpose of this experiment was to determine whether access to official information improved Amplify's answers and whether those answers could be trusted without checking the original documents.
Part 1: Asking Amplify Without Providing the Policies
In the first conversation, I did not provide Amplify with the official policy documents. Instead, I asked it to make its best guesses about remote-work eligibility, approval requirements, employee responsibilities, termination of remote-work arrangements, and the differences between WCU's policy and North Carolina's state telework policy.
Amplify frequently used words such as "likely," "probably," and "possibly." It suggested that employees might need positions that could be performed remotely, satisfactory performance records, completed training, and supervisor approval. It also guessed that first-year or probationary employees might be excluded.
For approval requirements, Amplify suggested that a direct supervisor would probably approve the arrangement, with possible involvement from a department head, IT, or Human Resources. It also guessed that employees would need to protect confidential information, use approved security tools, maintain productivity, protect university equipment, and communicate regularly with supervisors.
When asked whether arrangements could be changed or ended, Amplify suggested that performance issues, security violations, and operational needs could be reasons. Finally, it guessed that WCU's policy would resemble North Carolina's state telework policy because WCU is part of the UNC System, although WCU's rules might be less rigid.
Although these answers sounded reasonable, Amplify was making assumptions about WCU's actual requirements. I needed to compare its responses with the official policy documents before deciding whether the information was accurate.
Part 2: Comparing Amplify's Initial Answers with the Policies
The following table summarizes Amplify's first responses and identifies what must be checked against the official policy documents. The ratings are preliminary because the actual Canvas policy documents have not yet been independently verified.
| Question | Amplify's Answer | What the Policy Says | My Evaluation |
|---|---|---|---|
| 1. Who is eligible for remote work? | Employees probably need jobs that can be done remotely, satisfactory performance, training, and supervisor approval. Amplify also suggested that first-year or probationary employees might be excluded. | The later Amplify response, based on general WCU Flexible Work Guidelines, identified remote-work suitability, measurable performance, attendance, and communication as relevant considerations. It also stated that not all positions or organizational units qualify. These details need to be checked against the actual WCU IT Division Remote Work Standard. | Partially Accurate — provisional. Job suitability and performance are plausible criteria, but the suggestion that first-year or probationary employees might be excluded was a guess and requires evidence. |
| 2. Who must approve a remote-work arrangement? | A direct supervisor probably gives initial approval, while a department head, IT, or Human Resources might also be involved. | The later response described supervisor and division-head approval for ongoing arrangements lasting four months or more, with supervisors potentially approving shorter arrangements. This must be verified against the applicable policy and the specific IT Division approval process. | Partially Accurate — provisional. The answer correctly recognized that approval is required, but it did not initially establish the actual decision-makers or approval levels. |
| 3. What responsibilities does a remote employee have? | Employees probably need to protect confidential information, use approved security tools, maintain productivity, protect university equipment, work their scheduled hours, and communicate with supervisors. | The later response identified performance, attendance, work hours, availability, data security, equipment protection, recordkeeping, policy compliance, and communication. These requirements should be checked against the actual policy language. | Partially Accurate — provisional. Several responsibilities are plausible and appear in the later response, but the first answer did not establish which requirements were explicitly stated in the policy. |
| 4. Can a remote-work arrangement be changed or ended? | Amplify guessed that arrangements could be changed or ended because of performance problems, security violations, or operational needs, possibly with advance notice. | The later response described management discretion, modification and termination procedures, documentation, and a standard notice period of 10 business days, with possible exceptions. The notice period and other conditions require confirmation in the official agreement. | Partially Accurate — provisional. The general idea may be correct, but the first response did not establish the actual procedures, circumstances, or notice requirements. |
| 5. How does WCU's policy compare with North Carolina's state telework policy? | Amplify guessed that the policies would have similar eligibility, approval, documentation, and security requirements, but that WCU might be less rigid. | The later Amplify comparison identified differences involving duration thresholds, approval processes, eligibility, compensation and benefits, equipment, and out-of-state work. The actual WCU and North Carolina documents must be compared to confirm these differences. | Unsupported — provisional. The initial claim that WCU's policy was less rigid was not supported by evidence. Similarities between the policies also need to be demonstrated using the documents. |
Analysis of the Initial Answers
The initial responses show how an AI tool can make plausible guesses without knowing whether they are supported by a particular policy. Some of Amplify's suggestions may align with common workplace practices, but that does not mean WCU has adopted every suggested requirement.
For example, requiring employees to protect confidential information is a reasonable expectation in many workplaces. However, the initial response did not establish exactly which security practices WCU requires of remote employees. Similarly, Amplify's suggestion that probationary employees might be excluded was not enough to prove that such a restriction exists.
It is important to distinguish between an inaccurate answer and an unsupported answer. An inaccurate answer contradicts the policy, while an unsupported answer contains a claim that cannot be established from the available evidence. A claim should not be treated as correct simply because it sounds reasonable.
Part 3: Giving Amplify the Authoritative Sources
In a separate conversation, I provided policy-related source material and asked Amplify questions about remote work again. The later response was more detailed and included specific eligibility considerations, approval procedures, employee responsibilities, and conditions for modifying or ending an arrangement.
Amplify's later answer also noted that the provided documents did not include a specific WCU IT Division policy and that its eligibility response was based on the general WCU Flexible Work Guidelines. This distinction is important because a general university guideline may not contain every requirement of an IT Division-specific standard.
The later response discussed the following areas:
- Eligibility: Whether work can be completed remotely, whether performance can be measured, whether attendance is satisfactory, and whether effective communication can be maintained.
- Approval: Supervisor involvement and division-head approval for ongoing arrangements, with a different process potentially applying to temporary arrangements.
- Employee responsibilities: Meeting performance expectations, maintaining availability, protecting university information and equipment, following university policies, and communicating effectively.
- Modification and termination: Management discretion, documentation of changes, performance concerns, operational needs, and possible notice requirements.
- Comparison with state policy: Differences in approval processes, duration thresholds, equipment responsibilities, eligibility, and out-of-state work.
These responses offered more information to investigate than the original guesses. However, the details still need to be compared with the exact documents provided in Canvas before they can be considered verified policy requirements.
Did the Source Material Make a Difference?
Comparing the two conversations showed that the information available to Amplify affected the specificity of its responses. In the first conversation, Amplify relied on general guesses. In the later conversation, it gave more detailed explanations that could be compared with policy language.
One example involved eligibility for remote work. Initially, Amplify guessed that employees might need satisfactory performance, completed training, and supervisor approval. It also speculated that first-year or probationary employees might be excluded. The later response described more specific considerations, including whether the job could be performed remotely, whether performance could be measured, and whether the employee could maintain effective communication. It also acknowledged that not all positions or organizational units qualify. This was more detailed, although the specific requirements still needed to be verified against the correct policy.
A second example involved approval procedures. In the first conversation, Amplify guessed that a supervisor, department head, IT, or Human Resources might be involved. In the later conversation, it distinguished between ongoing arrangements lasting four months or more and shorter temporary arrangements. This was more specific because it described different approval processes rather than listing possible decision-makers. However, the actual approval requirements still needed to be confirmed using the applicable policy.
A third example involved changing or ending remote-work arrangements. The first response generally suggested that performance issues, security violations, or operational needs could lead to termination. The later response discussed modification procedures, documentation, management discretion, and a standard notice period. This additional detail could help an employee understand the process, but the notice period and conditions must be confirmed in the official documents.
Overall, the later answers were more detailed and provided more specific information to check. However, more detail does not automatically mean greater accuracy. The later response also acknowledged that it was using general WCU Flexible Work Guidelines rather than a specific IT Division policy. This showed that providing documents can improve an AI's answers, but users must still make sure the AI is relying on the correct source and representing it accurately.
Part 4: What Does This Mean for AI at Work?
1. Would I Trust the AI?
I would not rely only on an AI-generated answer to decide whether I am allowed to work remotely at WCU. My experiment showed that Amplify could produce reasonable-sounding answers without having the actual policy documents. Some of its suggestions might be correct, but others could be incomplete or unsupported.
If I made a workplace decision based on an incorrect answer, I could misunderstand the university's requirements or fail to follow the proper approval process. I would use AI to help explain a policy or identify important requirements, but I would verify the answer against the official policy or contact the appropriate university office before making a decision.
2. Information vs. Action
Giving an AI access to information is different from giving it the ability to take action. Access to information allows an AI to read documents, answer questions, summarize policies, and make recommendations. Giving an AI the ability to take action means it can do something in the workplace, such as send an email, submit a request, update a record, or change information in a system.
This difference matters because an incorrect answer and an incorrect action can have different consequences. If an AI summarizes a policy incorrectly, a person can check the original document before relying on the summary. If an AI can also submit forms or change records, its mistake could directly affect employees or university operations.
For this reason, workplaces should establish appropriate permissions and review procedures. Actions that have important consequences should include human oversight or confirmation, especially when they affect employee records, workplace arrangements, or compliance with university policies.
3. Where Would You Draw the Line?
One workplace task I would be comfortable allowing an AI assistant to perform with little supervision is organizing meeting notes into a summary and a list of action items. This task can save time, and mistakes can usually be corrected by reviewing the summary.
An example where I would want a human to verify the AI's work is determining whether an employee qualifies for remote work or submitting a request that affects an employee's official work arrangement. These decisions involve university rules and can affect an employee's working conditions. A human should verify the applicable policy, confirm that the information is correct, and ensure the required approvals are obtained before anything is finalized.
I would treat these tasks differently because the consequences of an error are different. Organizing meeting notes is mainly an administrative convenience, while interpreting employment policies or changing an employee's work arrangement requires accuracy and accountability. AI can help with both tasks, but the level of human oversight should depend on the risks involved.
Conclusion: Can You Trust AI at Work?
This experiment helped me understand that AI can be useful without always being reliable. When Amplify did not have the policy documents, it relied on guesses and made suggestions that sounded reasonable but were not necessarily supported by WCU's actual rules. In the later conversation, it provided more detailed responses, but those details still needed to be verified against the appropriate source documents.
AI can help employees understand policies, summarize information, and complete routine tasks more efficiently. However, access to information does not guarantee that an AI will interpret it correctly, and the ability to take action creates additional risks when mistakes could have real consequences.
My main takeaway is that AI should be used as an assistant rather than as the final authority. For routine tasks, it can save time and improve efficiency. For important workplace decisions, employees should check authoritative sources and ensure that humans remain responsible for verifying the information and approving consequential actions.