[squid-users] Assert(call->dialer.handler == callback)

Thu Apr 30 16:14:55 UTC 2015

I've just migrated a system from Squid 3.4.10 to 3.5.3 and I'm getting 
frequent crashes with an assertion of "call->dialer.handler == callback" 
in Read.cc:comm_read_cancel().

call->dialer.handler == (IOCB *) 0x7ffe1493b2d0 
<TunnelStateData::ReadClient(Comm::ConnectionPointer const&, char*, 
size_t, Comm::Flag, int, void*)>

callback == <IdleConnList::Read(Comm::ConnectionPointer const&, char*, 
size_t, Comm::Flag, int, void*)>

This is quite a busy system doing server-first ssl_bump and I get a lot 
of SSL negotiation errors in cache.log (these were present under 3.4.10 
too).  I think a good chunk of these are Team Viewer, which abuses 
CONNECTs to port 443 of remote servers to do non-SSL traffic, so 
obviously isn't going to work with ssl_bump.  I _suspect_ that the 
assertion may be being triggered by these SSL errors (e.g. connection 
being unexpectedly torn down because SSL negotiation failed?), but I 
can't easily prove that.

I don't quite understand the comm_read_cancel() function though - as far 
as I can see, the callback parameter is only used in the assert() - is 
that correct?

Stack trace:
#0  0x00007ffe1155d625 in raise (sig=6) at 
../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007ffe1155ee05 in abort () at abort.c:92
#2  0x00007ffe148210df in xassert (msg=Unhandled dwarf expression opcode 
0xf3
) at debug.cc:544
#3  0x00007ffe14a62787 in comm_read_cancel (fd=600, 
callback=0x7ffe148c8dd0 <IdleConnList::Read(Comm::ConnectionPointer 
const&, char*, size_t, Comm::Flag, int, void*)>,
     data=0x7ffe176c8298) at Read.cc:204
#4  0x00007ffe148c5e62 in IdleConnList::clearHandlers 
(this=0x7ffe176c8298, conn=...) at pconn.cc:157
#5  0x00007ffe148c94ab in IdleConnList::findUseable 
(this=0x7ffe176c8298, key=...) at pconn.cc:269
#6  0x00007ffe148c979d in PconnPool::pop (this=0x7ffe145db010, dest=..., 
domain=Unhandled dwarf expression opcode 0xf3
) at pconn.cc:449
#7  0x00007ffe14852142 in FwdState::pconnPop (this=Unhandled dwarf 
expression opcode 0xf3
) at FwdState.cc:1153
#8  0x00007ffe14855605 in FwdState::connectStart (this=0x7ffe2034c4e8) 
at FwdState.cc:850
#9  0x00007ffe14856a31 in FwdState::startConnectionOrFail 
(this=0x7ffe2034c4e8) at FwdState.cc:398
#10 0x00007ffe148d2fa5 in peerSelectDnsPaths (psstate=0x7ffe1fd0c028) at 
peer_select.cc:302
#11 0x00007ffe148d6a1d in peerSelectDnsResults (ia=0x7ffe14f0ac20, 
details=Unhandled dwarf expression opcode 0xf3
) at peer_select.cc:383
#12 0x00007ffe148a8e71 in ipcache_nbgethostbyname (name=Unhandled dwarf 
expression opcode 0xf3
) at ipcache.cc:518
#13 0x00007ffe148d23c1 in peerSelectDnsPaths (psstate=0x7ffe1fd0c028) at 
peer_select.cc:259
#14 0x00007ffe148d6a1d in peerSelectDnsResults (ia=0x7ffe14f0ac20, 
details=Unhandled dwarf expression opcode 0xf3
) at peer_select.cc:383
#15 0x00007ffe148a8e71 in ipcache_nbgethostbyname (name=Unhandled dwarf 
expression opcode 0xf3
) at ipcache.cc:518
#16 0x00007ffe148d23c1 in peerSelectDnsPaths (psstate=0x7ffe1fd0c028) at 
peer_select.cc:259
#17 0x00007ffe148d382b in peerSelectFoo (ps=0x7ffe1fd0c028) at 
peer_select.cc:522
#18 0x00007ffe149bba6a in ACLChecklist::checkCallback 
(this=0x7ffe2065b9e8, answer=...) at Checklist.cc:167
#19 0x00007ffe148d3f5a in peerSelectFoo (ps=0x7ffe1fd0c028) at 
peer_select.cc:459
#20 0x00007ffe148d5176 in peerSelect (paths=0x7ffe2034c540, 
request=0x7ffe1b660b70, al=Unhandled dwarf expression opcode 0xf3
) at peer_select.cc:163
#21 0x00007ffe14852ae3 in FwdState::Start (clientConn=..., 
entry=0x7ffe1b0da790, request=0x7ffe1b660b70, al=...) at FwdState.cc:366
#22 0x00007ffe14801401 in clientReplyContext::processMiss 
(this=0x7ffe1fcf5838) at client_side_reply.cc:691
#23 0x00007ffe14801eb0 in clientReplyContext::doGetMoreData 
(this=0x7ffe1fcf5838) at client_side_reply.cc:1797
#24 0x00007ffe14805a89 in ClientHttpRequest::httpStart 
(this=0x7ffe1dcda618) at client_side_request.cc:1518
#25 0x00007ffe14808cac in ClientHttpRequest::processRequest 
(this=0x7ffe1dcda618) at client_side_request.cc:1504
#26 0x00007ffe14809013 in ClientHttpRequest::doCallouts 
(this=0x7ffe1dcda618) at client_side_request.cc:1830
#27 0x00007ffe1480b453 in checkNoCacheDoneWrapper (answer=..., 
data=0x7ffe1e5db378) at client_side_request.cc:1400
#28 0x00007ffe149bba6a in ACLChecklist::checkCallback 
(this=0x7ffe1c88b4a8, answer=...) at Checklist.cc:167
#29 0x00007ffe1480b40a in ClientRequestContext::checkNoCache 
(this=0x7ffe1e5db378) at client_side_request.cc:1385
#30 0x00007ffe14809c04 in ClientHttpRequest::doCallouts 
(this=0x7ffe1dcda618) at client_side_request.cc:1748
#31 0x00007ffe1480d109 in ClientRequestContext::clientAccessCheckDone 
(this=0x7ffe1e5db378, answer=Unhandled dwarf expression opcode 0xf3
) at client_side_request.cc:821
#32 0x00007ffe1480d898 in ClientRequestContext::clientAccessCheck2 
(this=0x7ffe1e5db378) at client_side_request.cc:718
#33 0x00007ffe14809767 in ClientHttpRequest::doCallouts 
(this=0x7ffe1dcda618) at client_side_request.cc:1721
#34 0x00007ffe1480afca in ClientHttpRequest::handleAdaptedHeader 
(this=0x7ffe1dcda618, msg=Unhandled dwarf expression opcode 0xf3
) at client_side_request.cc:1935
#35 0x00007ffe14abbcaa in JobDialer<Adaptation::Initiator>::dial 
(this=0x7ffe1ce04990, call=...) at ../../src/base/AsyncJobCalls.h:174
#36 0x00007ffe149bea69 in AsyncCall::make (this=0x7ffe1ce04960) at 
AsyncCall.cc:40
#37 0x00007ffe149c272f in AsyncCallQueue::fireNext (this=Unhandled dwarf 
expression opcode 0xf3
) at AsyncCallQueue.cc:56
#38 0x00007ffe149c2a60 in AsyncCallQueue::fire (this=0x7ffe16f70bf0) at 
AsyncCallQueue.cc:42
#39 0x00007ffe1484110c in EventLoop::runOnce (this=0x7fffcb8c4be0) at 
EventLoop.cc:120
#40 0x00007ffe148412c8 in EventLoop::run (this=0x7fffcb8c4be0) at 
EventLoop.cc:82
#41 0x00007ffe148ae191 in SquidMain (argc=Unhandled dwarf expression 
opcode 0xf3
) at main.cc:1511
#42 0x00007ffe148af2e9 in SquidMainSafe (argc=Unhandled dwarf expression 
opcode 0xf3
) at main.cc:1243
#43 main (argc=Unhandled dwarf expression opcode 0xf3
) at main.cc:1236

(sorry about the DWARF errors - it looks like I've got a version 
mismatch between gcc and gdb)

-- 
  - Steve Hill
    Technical Director
    Opendium Limited     http://www.opendium.com

Direct contacts:
    Instant messager: xmpp:steve at opendium.com
    Email:            steve at opendium.com
    Phone:            sip:steve at opendium.com

Sales / enquiries contacts:
    Email:            sales at opendium.com
    Phone:            +44-1792-824568 / sip:sales at opendium.com

Support contacts:
    Email:            support at opendium.com
    Phone:            +44-1792-825748 / sip:support at opendium.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: steve.vcf
Type: text/x-vcard
Size: 283 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150430/b4213adb/attachment.vcf>